Gobinath Raju
Powder Springs,GA
Summary
Senior Google Cloud and Big Data Engineer with over 14 years of experience in leading the design of distributed systems and implementing AI/ML solutions. Proven track record in driving digital transformations and architecting enterprise-grade GenAI chatbots using Google ADK, Vertex AI, Bigtable, and BigQuery, as well as migrating over 1,000 legacy jobs to GCP. Expertise in aligning business requirements with high-performance execution, leveraging extensive knowledge in Spark, Dataproc, Dataflow, Hadoop, MapReduce, Hive, and Impala to create scalable, customer-centric cloud architectures that enhance operational productivity. Committed to delivering innovative solutions that drive organizational growth and efficiency.
Overview
14
14
years of professional experience
1
1
Certification
Work History
Senior Software Engineer
The Home Depot
Atlanta, GA, USA
12.2023 - Current
- AI-ML Solution Architecture: Architected and developed an intelligent chatbot platform leveraging Google Agent Development Kit (ADK), utilizing BigTable for low-latency chat history and BigQuery for robust monitoring, logging, and auditing dashboards.
- MLOps & Deployment: Led the end-to-end Machine Learning Operations (MLOps) lifecycle, including model training and deployment using Vertex AI to inform real-time decision-making for A/B testing and test groups.
- Reduced monthly BigQuery expenditure by 30% by implementing table partitioning and clustering on high-volume datasets, minimizing daily data scans by 15TB.
- Reduced Bigtable operational costs by optimizing row key designs to eliminate hotspots, allowing for a 30% reduction in required cluster nodes and designed with GC policy (age & versions) removed older/unused rowkeys.
- Improved pipeline cost-efficiency by 20% through the implementation of Vertical Autoscaling and custom worker machine types tailored to memory-intensive transformations.
- Automated GitHub and Airflow maintenance pipelines to prune stale branches
- Data Pipeline Engineering: Designed and implemented scalable data ingestion strategies, including a Dataflow batch job to load data into BigTable and BigQuery, and a Dataflow streaming pipeline for real-time audit, customer feedback, and application logs.
- API & Microservices Development: Developed high-performance, low-latency Python FastAPI microservices, implementing background tasks to ensure minimal response times and seamless application performance.
- DevOps & CI/CD: Established robust Continuous Integration/Continuous Deployment (CI/CD) pipelines using GitHub Actions and the internal deployment platform, Vulcan, ensuring efficient and automated software delivery.
- Monitoring & Optimization: Proactively monitored production Service Level Objectives (SLOs), focusing on performance tuning and cost optimization across code, infrastructure, and data processing to maintain high availability and reliability.
GCP Data Engineer
Arohak Inc
Atlanta, GA, USA
10.2022 - 12.2023
- Migrated over 1000 jobs across 4 modules from Hive, Teradata, Dataproc, and BigQuery jobs to Composer DAG using Python. Developed 3 cloud functions for data preprocessing and loading.
- Created 50 ingestion jobs to handle various file types including text, CSV, ORC, Parquet, tar.gz, and .z files.
- Converted 300 shell script jobs to run in Composer by transforming them into JSON format using cloud functions.
- Built Python utilities and PySpark jobs for data migration, comparison, and deployment tasks.
- Developed shell script utilities for loading data into BigQuery and comparing counts and data between current and new production tables. Designed utilities for migrating data from Hive tables to BigQuery tables.
- Quickly acquired knowledge of all jobs and provided support across development, testing, UAT, and production environments.
Big Data developer
Mindtree Ltd
Atlanta, GA, USA
05.2020 - 09.2022
- Distributed computing with large data sets using the Bigtable, BigQuery, Datastore, Dataflow, Spark Java, Cassandra, Kubernetes Cluster, Docker and Dataproc
- Exploring new technologies to do POC on Feast, Google Feature store(Vertex AI) and micronaut microservices
- Working with Google team and developing a POC to test and use their Vertex-AI Feature Store for Home Depot.
- Developed a Data Quality Framework to identify good and bad data from the large datasets, prepare Data Quality metrics for users for each set of inputs, enabled to support multiple sources of inputs
- Working on performance test using NeoLoad & ReLoad apps to share the performance report for each of the POC we developed.
- Created a Data migration framework to migrate the data from Legacy environment to Google Cloud Cloud using Spark scala
- Developed an E2E application using micronaut and deployed on Google cloud Kubernetes using Docker
- Setup and deploy the GitHub projects using TeamCity, Spinnaker, Docker and Kubernetes Engine in GCP
- Setup pipeline in concourse to automate the built and deploy the projected related scripts, jars and configurations
CICD activities such as Development, Stage, Prod and Deploy in multiple environments with automated scripts
Big Data Developer
Galax-Esystems Corporation
Cary, NC, USA
11.2019 - 04.2020
- Understand the requirement, Design and implement in the most recent Big Data technologies with the complex methodologies.
- Distributed computing with large data sets using the Spark Scala (SQL), Python, and Hive.
- Migrating on-premises applications to Amazon Cloud.
- Rewriting legacy application to Spark Scala.
- Set application, User policies using Apache Ranger.
- Developing util, ad hoc scripts using Python, Shell and Spark.
- Data Catalyst is the Successful businesses effectively leverage data to shape strategy, inform decisions, identify opportunities, and deliver time-to-business value.
Consultant
Deloitte Consulting LLP
Atlanta, GA, USA
08.2018 - 09.2019
- Developed a POC by using BigQuery and GCP tools to do the Automated Legacy Database Migration to GCP with two simple steps using Google products.
- Created a stand-alone application in Spark Scala to migrate the entire schema or list of tables from any traditional databases to Cloud or Hadoop ecosystems.
- Design database queries, triggers, procedures, functions and packages for reporting and data analytics.
- Have done GCP Certifications on Coursera and Qwiklabs.
- Hands on experience with GCP (Google Cloud Platform) products like Dataproc, BigQuery, BigTable, Postgres and Google Cloud storage.
Technical Lead (On shore)
Tata Consultancy Services
Atlanta, GA, USA
11.2016 - 07.2018
- Understand the requirement, Design, and implement in the most recent Big Data technologies with the complex methodologies.
- Develop plans/projects from conceptualization to implementation.
- Distributed computing with large data sets using the Spark Scala, MapReduce, Impala and Hive.
- AWS S3 Integration, storage processing, spin up cluster and set up clusters quickly using automated shell scripts.
- Rewrite Impala and Hive modules to Spark Scala.
- CICD activities such as Development, Test, Compare and Deploy in multiple environments with automated scripts.
- Work with DSci team, understand the requirement, prepare design documents, implement the complex methodology. Remove RDBMS dependencies and rewrite every module to work in Cloud platform.
- Coordinate with Clients, DSci team, Agile team, offshore team, upstream team, downstream team to meet the deliverables on time.
- Work closely to meet the deliverables on time with multiple shifts by coordinating the Offshore (India).
Technical Lead (Offshore)
Tata Consultancy Services
Chennai, India
01.2012 - 10.2016
- Rewrite Java, Oracle application to Netezza to work with distributed RDBMS databases.
- POC to migrate legacy applications into Bigdata technologies such as MapReduce and Hive and prove the performance with big data datasets and statistics.
- Gather requirement, designing, planning and implementation.
- Develop the code from HLD, LLD to software application.
- Closely work on Production support, Deployments, Deliverables.
- Integrate SQOOP to extract the data from legacy systems.
- Support Java, Oracle, Netezza, Sybase applications in production system.
Education
Bachelor of Engineering - Electronics and Communication Engineering
M. Kumarasamy College of Engineering
Karur05.2011
Skills
- Google Cloud Platform
- Google ADK
- BigQuery
- BigTable
- Dataflow
- Datastore
- Pub/Sub
- Composer (AirFlow)
- Dataproc
- GCS
- Compute Engine
- Gemini Models (flash25)
- Prompt Tuning
- VertexAI
- JAVA
- Python
- SQL
- NoSQL
- Scala
- Model Training (Reinforcement Learning)
- Microservices/Frameworks
- Python FastAPI
- Micronaut
- Apache Spark Java & PySpark
- MapReduce & Apache Hive
- Hadoop & HDFS Ecosystem
- Cloudera Impala
- Sqoop
- Unix Shell scripting
- Cassandra
Certification
- Oracle Certified Professional - Java 6 (OCJP 6.0)
- Data Engineering on Google Cloud Platform (Coursera) and Qwiklabs
Languages
English
Professional Working
Tamil
Native or Bilingual
Timeline
Senior Software Engineer
The Home Depot
12.2023 - Current
GCP Data Engineer
Arohak Inc
10.2022 - 12.2023
Big Data developer
Mindtree Ltd
05.2020 - 09.2022
Big Data Developer
Galax-Esystems Corporation
11.2019 - 04.2020
Consultant
Deloitte Consulting LLP
08.2018 - 09.2019
Technical Lead (On shore)
Tata Consultancy Services
11.2016 - 07.2018
Technical Lead (Offshore)
Tata Consultancy Services
01.2012 - 10.2016
Bachelor of Engineering - Electronics and Communication Engineering
M. Kumarasamy College of Engineering
Personal Information
- Date of Birth: 1989-08-13
- Nationality: Indian
- Work Permit: H1-B(I-140) Valid till Nov 2026
- ID Type: Passport
- ID Number: V2266110