Summary
Overview
Work History
Education
Skills
Websites
Certification
Timeline
Generic

RAJESH RAYIDI

RICHMOND,TX

Summary

Detail-oriented and highly analytical data professional with a strong background in data analytics, specializing in optimizing Apache Spark jobs to enhance performance and resource efficiency. Designing and implementing scalable data pipelines using GCP services such as BigQuery, Dataflow, Dataproc, and cloud storage. Adept at applying advanced optimization techniques-including partitioning, caching, and memory tuning, streaming data processing workflows, and reducing costs in cloud environments.

Overview

8
8
years of professional experience
1
1
Certification

Work History

GCP Data Engineer

NBC Universal
04.2024 - Current
  • Developed and optimized batch and real-time pipelines using GCP services and Spark Scripts to process structured and unstructured datasets from API’s internal systems and external platforms.
  • Streamlined large-scale data ingestion and transformations using Spark and Dataflow to enable real-time viewer analytics and reporting for scheduling, ad targeting, and audience insights.
  • Integrated Apache Kafka, Spark Structured Streaming, and Pub/Sub for low-latency ingestion, and designed a high-performance warehouse in BigQuery.
  • Implemented ML models with Vertex AI for versioning, continuous training, and automated deployment.
  • Leveraged Spark on Dataproc, and CI/CD (cloud functions, GitHub) with monitoring for scalable and reliable data & ML operations.
  • Automated infrastructure deployment with Terraform and strengthened governance by enforcing IAM, Data Catalog, and Collibra compliance (GDPR).
  • Delivered stakeholder-facing dashboards in Looker and Power BI with KPI’s like ad revenue, viewer trends, and ROI to drive strategic decisions.

Data Engineer

RWE Clean Energy
11.2021 - 02.2024
  • Improved the efficiency of Spark scripts by using optimization techniques.
  • Processed renewable energy data from SCADA systems, weather feeds, and trading platforms using AWS Glue, Spark, S3.
  • Optimized data transformation and storage with Spark, MySQL, and Redshift to support high-performance analytics and operational reporting.
  • Automated orchestration with Airflow (MWAA) and implemented real-time ingestion.
  • Streamlined deployments through CI/CD with Terraform and GitHub, ensuring reliability and consistency across environments.
  • Build dynamic dashboards in Looker, providing actionable insights on energy output, grid performance, and asset health.

Data Analyst

GlobalLogic
, India
06.2017 - 12.2019
  • Company Overview: GlobalLogic, India
  • Collected and cleaned data from ERP systems, IoT devices, and quality control databases using SQL and Python.
  • Removed duplicates, fixing errors, and fixing missing values to support analysis.
  • Create easy-to-understand dashboards and reports using Power BI to help monitor machine health and production problems.
  • Worked with teams like production, supply chain, and quality control to automate routine reports and improve data flow.
  • GlobalLogic, India

Education

Bachelor of Technology - Information Technology

Hindustan Institute of Technology and Sciences
Chennai, IND

Skills

  • Python
  • Java
  • SQL
  • Scala
  • MySQL
  • Cassandra
  • Teradata
  • Dynamo DB
  • HDFS
  • MapReduce
  • PySpark
  • Spark Streaming
  • Spark SQL
  • Hive
  • Sqoop
  • Airflow
  • Snowflake
  • BigQuery
  • GCP
  • AWS- EC2
  • EMR
  • S3
  • Redshift
  • Glue
  • Lambda
  • Tableau
  • Power BI
  • Data Structures and Algorithms
  • Object Oriented Programming
  • Parallel Programming
  • Data pipeline development
  • Real-time analytics
  • Machine learning deployment
  • Cloud infrastructure automation
  • BigQuery optimization
  • Data governance compliance

Websites

Certification

Google Cloud Platform (GCP) certified professional data engineer

Timeline

GCP Data Engineer

NBC Universal
04.2024 - Current

Data Engineer

RWE Clean Energy
11.2021 - 02.2024

Data Analyst

GlobalLogic
06.2017 - 12.2019

Bachelor of Technology - Information Technology

Hindustan Institute of Technology and Sciences

Similar Profiles

CREATE PROFILE
RAJESH RAYIDI