Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Dhanush Reddy

Dallas,USA

Summary

  • A Google Certified Professional Data Engineer with 7+ years of experience in data engineering, cloud data platforms, and analytics across diverse domains. Extensive hands-on experience with GCP services such as BigQuery, Cloud Dataflow, Cloud Pub/Sub, Cloud Storage, Dataproc, Composer (Airflow), and Cloud Run for scalable data processing and orchestration.
  • Designed and deployed secure enterprise APIs using Apigee and Cloud Run, enabling real-time access to customer data while ensuring security and compliance. Proficient in Python (Pandas, NumPy, PySpark, Apache Beam), SQL (Presto, Hive, BigQuery Standard SQL), and Spark for developing ELT pipelines and advanced analytics workflows.
  • Developed streaming and batch pipelines using Apache Beam on Dataflow, integrating real-time and historical data into BigQuery. Automated infrastructure provisioning using Terraform, ensuring scalable, secure, and auditable deployments across multiple GCP projects.
  • Strong experience building CI/CD pipelines using Azure DevOps, Buildkite, and Git, including unit testing for DAGs and data validation in Python.
  • Migrated legacy ETL systems from Oracle and SAS to GCP, leveraging Dataproc, BigQuery, and Cloud Functions for cost-effective performance. Experienced in working with Cloud Composer (Airflow as a service), using a variety of operators (Bash, PythonCallable, Branching, Hadoop) for robust pipeline orchestration.
  • Expertise in data visualization and reporting using Tableau, Looker Studio, and Power BI, including performance tuning and dashboard optimization. Implemented data validation frameworks to compare source files with BigQuery tables and ensure high data quality across ingestion workflows.
  • Built ML pipelines on Vertex AI for customer sentiment analysis and forecasting, and developed end-to-end MLOps workflows integrated with production data. Knowledge of Kubernetes, Stackdriver Logging, and Monitoring, with practical experience in setting up alerting and log routing for GCP resources.
  • Strong knowledge of Unix/Linux systems, with experience in Bash/Shell scripting for automation, job scheduling, and data movement tasks. Collaborated with security teams to set up custom IAM roles, firewall rules, and vulnerability scans using tools like Forseti.
  • Enthusiastic about learning and adopting new technologies introduced in GCP, with a strong focus on cost efficiency, performance, and governance.

Overview

7
7
years of professional experience
1
1
Certification

Work History

Sr. GCP Data Engineer

Wendy’s
Remote, USA
01.2022 - Current
  • Designed and deployed scalable GCP data pipelines to ingest Gateway data using Cloud Run and Apigee API endpoints for real-time processing.
  • Engineered secure enterprise APIs with Apigee and Cloud Run to provide controlled access to customer data.
  • Developed streaming and batch pipelines with Apache Beam and Dataflow, sourcing data from Pub/Sub and other connectors.
  • Automated ingestion of Google Reviews into BigQuery, supporting analytics for over 1,000 business locations.
  • Led the development of a Form Integration system for WDRV (Wendy’s Done Right Visit) KPIs, building ETL models to power MoE dashboards.
  • Orchestrated robust ETL workflows via Cloud Composer, ensuring operational reliability and performance.
  • Centralized data from multiple systems into BigQuery with data quality checks and validation frameworks.
  • Improved query performance and reduced costs through BigQuery table optimization (partitioning, clustering).
  • Built ML pipelines for customer sentiment analysis and forecasting using Vertex AI and MLOps workflows.
  • Created POCs for end-to-end ML workflows on Vertex AI Pipelines integrated with production data.
  • Managed infrastructure provisioning across GCP using Terraform for scalable and secure operations.

GCP Data Engineer

Schlumberger
Houston, TX
11.2019 - 11.2022
  • Maintaining the infrastructure in multiple projects across the organization in Google cloud platform using terraform (Infrastructure as code).
  • Making existing 'Bigquery with Tableau for reporting' more performant using various techniques like partitioning the right column and testing the solutions using different scenarios.
  • Developed ELT processes from the files from abinitio, google sheets in GCP with compute being dataprep, dataproc (pyspark) and Bigquery.
  • Good knowledge in building data pipelines in airflow as a service (composer) using various operators.
  • Build a program using Python and Apache beam to execute it in cloud Dataflow and to run Data validation jobs between raw source file and big query tables.
  • Extensive use of cloud shell SDK in GCP to configure/deploy the services like Cloud Dataproc (Managed Hadoop), Google Cloud Storage and Cloud Bigquery.
  • Created BigQuery jobs for loading the data into BigQuery tables from data files stored in Google Cloud storage daily.
  • Developed report using Tableau which keeps track of the dashboards published to Tableau Server, which help us find the potential future clients in the organization.
  • Helped teams to identify the bigquery usage patterns and tuned bigquery queries fired from dataflow jobs and more with respect to how app teams can use the bigquery tables for store level attributes.
  • Migrated an Oracle SQL ETL to run on google cloud platform using cloud dataproc & bigquery, cloud pub/sub for triggering the airflow jobs.
  • Worked on using presto, hive, spark-sql, bigquery using python client libraries and building interoperable and faster programs for analytics platforms.
  • Hands on experience in using all the big data related services in Google Cloud Platform.
  • Used apache airflow in GCP composer environment to build data pipelines and used various airflow operators like bash operator, Hadoop operators and python callable and branching operators.
  • Developed new techniques for orchestrating the Airflow built pipelines and used airflow environment variables for defining project level and encrypting the passwords.
  • Working knowledge in working around Kubernetes in GCP, working on creating new monitoring techniques using the stackdriver’s log router and designing reports in data studio.
  • Served as an integrator between data architects, data scientists, and other data consumers.
  • Converted SAS code to python/spark-based jobs in cloud dataproc/big query in GCP.
  • Moved Data between bigquery and Azure Data Warehouse using ADF and created Cubes on AAS with lots of complex DAX language for memory optimization for reporting.
  • Used cloud pub/sub and cloud functions for some specific use cases such as triggering workflows upon messages.
  • Development of data pipelines with cloud composer for orchestrating, cloud dataflow for building scalable machine learning algorithms for clustering, cloud data prep for exploration.
  • Migrated previously written cloud prep jobs to Bigquery.
  • Experience in using/setting up Forseti for scanning threats in the projects.
  • Worked closely with security teams by providing them the logs with respect to firewalls, VPC’s and setting up rules in GCP for vulnerability.
  • Created custom roles for sandbox environments using terraform to avoid vulnerabilities.

Data Engineer

TechBridge Partners LLC
Hyderabad, India
05.2018 - 11.2019
  • Carried out data transformation and cleansing using SQL queries, Python and PySpark.
  • Was responsible for ETL and data validation using SQL Server Integration Services.
  • Built SQL queries to build the reports for presales and secondary sales estimations.
  • Hands-on experience with building data pipelines in python/Pyspark/HiveSQL/Presto.
  • Have used SAS for data analysis as well as Python for building ETL pipelines using pandas framework.
  • Converted previously written SAS programs into python for one the ETL project.
  • Carried out data transformation and cleansing using SQL queries, Python and Pyspark.
  • Worked on the backend using Python and Spark to perform several aggregation logics.
  • Created Oracle Stored Procedure to implement complex business logic for better performance.
  • Extensively used PL/SQL to build Oracle Reports 10g and views for processing data, enforcing referential integrity, and needed business rules.
  • Developing python programs that can run the end-to-end data migration and as well as transformation and load data into sinks such as oracle and MySQL.
  • Developed Python scripts to create data files from database and post them to FTP server on daily bases using windows task scheduler.

Education

Bachelor of Science - Computer Science

Geethanjali College of Engineering And Technology
India

Skills

  • MySQL
  • MS SQL Server
  • T-SQL
  • Oracle
  • PL/SQL
  • Teradata
  • Google Cloud Platform
  • GCP Cloud Storage
  • Big Query
  • Composer
  • Cloud Dataproc
  • Cloud SQL
  • Cloud Functions
  • Cloud Pub/Sub
  • Dataflow
  • Vertex AI
  • Apache Beam
  • Spark
  • Hadoop
  • Google Big Data stack
  • Azure Big Data Stack
  • DBT
  • SSIS
  • Power BI
  • Data Studio
  • Tableau
  • Pandas
  • SciPy
  • Numpy
  • Matplotlib
  • Google Cloud Python SDK
  • Shell/Bash
  • C#
  • R
  • Python
  • SQL

Certification

Google Cloud Certified Professional Data Engineer

Timeline

Sr. GCP Data Engineer

Wendy’s
01.2022 - Current

GCP Data Engineer

Schlumberger
11.2019 - 11.2022

Data Engineer

TechBridge Partners LLC
05.2018 - 11.2019

Bachelor of Science - Computer Science

Geethanjali College of Engineering And Technology

Similar Profiles

CREATE PROFILE
Dhanush Reddy