Surendra Reddy Medapati
Salt Lake City,Utah
Summary
Results-driven Data Engineer with 4+ years of experience designing and implementing scalable ETL pipelines, building cloud-based data platforms, and supporting cross-functional teams with real-time insights. Adept in Apache Spark, Python, Scala, and GCP/AWS ecosystems, with proven ability to improve data reliability, quality, and performance. Strong collaborator with experience supporting machine learning workflows, managing CI/CD pipelines, and optimizing large-scale data solutions to drive business value and operational efficiency.
Overview
5
5
years of professional experience
Work History
AI/ML Engineer
Cypress
09.2024 - Current
- Engineered Spark pipelines on GCP Dataproc to process 15M+ sensor records per week, reducing batch processing time by 28%.
- Automated pipeline orchestration with Airflow, increasing success rate of ML batch jobs from 93% to 99.7% uptime.
- Partnered with analysts to deliver curated BigQuery datasets, cutting query times by 40% and improving data trust.
- Built data quality validation layer using Python, reducing manual QA checks by 12 hours/week.
Data Engineer
TargetArc
06.2024 - 09.2024
- Developed ingestion pipelines in Scala and Airflow, improving throughput by 35% for healthcare data sources (processing ~200K rows/hour).
- Enabled real-time monitoring with Grafana, reducing incident response time by 50%.
- Integrated 10K+ biomedical entities into Redshift data lake, increasing coverage of LLM-based ML models by 20%.
- Improved end-to-end SLA adherence by building custom retry logic and checkpointing into Spark jobs.
Data Engineer
ProActive IT
01.2024 - 05.2024
- Migrated 50+ legacy ETL jobs into BigQuery and Airflow, cutting pipeline failures by 45% and reducing costs by $1,200/month.
- Tuned 10+ critical SQL queries used in financial dashboards, improving response time from 2 minutes to under 15 seconds.
- Processed 1.2M+ transaction rows monthly using GCS + BigQuery, ensuring data freshness for BI tools.
- Reduced manual deployment time by 70% by implementing CI/CD for pipeline delivery.
Data Analyst
Cognizant
08.2020 - 07.2022
- Led migration of 1,000+ SAS jobs to Redshift/Spark, improving ETL runtime by 40% and cutting operational overhead by ~$50K/year.
- Built Hive workflows on EMR to process ~5M healthcare claims/week for eligibility and payments processing.
- Used Icedq to validate 100% of migrated datasets across 3+ dev/test/prod environments, reducing QA escalations.
- Maintained 40+ scheduled jobs via Control-M, ensuring high availability and 99.9% SLA compliance.
Data Analyst
BSNL
01.2020 - 05.2020
- Designed Looker dashboards across 3 departments, reducing reporting turnaround time from 2 days to 3 hours.
- Migrated 200+ ETL objects from SQL Server to Snowflake, increasing report refresh speed by 60%.
- Queried over 50M rows monthly for operations metrics and built a SQL alerting system to catch anomalies in real-time.
- Reduced dependency on Excel reports by onboarding 10+ teams to centralized dashboards.
Education
Master of Science - Computer Science
Saint Leo University
Tampa, FL05-2024
Bachelor of Science - Computer Science
SASTRA University
India05-2020
Skills
- Scala
- Python
- SQL
- PySpark
- Apache Spark
- Spark Streaming
- Kafka
- Hadoop
- Hive
- GCP (BigQuery, GCS, Dataproc)
- AWS (S3, Glue, Lambda, Redshift)
- Snowflake
- Apache Airflow
- Control-M
- Docker
- GitLab CI/CD
- Star/Snowflake Schema
- Redshift
- BigQuery
- Data Lakes
- Power BI
- Grafana
- Looker
Certification
AWS Certified Architect
Timeline
AI/ML Engineer
Cypress
09.2024 - Current
Data Engineer
TargetArc
06.2024 - 09.2024
Data Engineer
ProActive IT
01.2024 - 05.2024
Data Analyst
Cognizant
08.2020 - 07.2022
Data Analyst
BSNL
01.2020 - 05.2020
Master of Science - Computer Science
Saint Leo University
Bachelor of Science - Computer Science
SASTRA University
Similar Profiles
Shakiria A. JonesShakiria A. Jones
Registered Nurse at CypressRegistered Nurse at Cypress
Nadege SmithNadege Smith
Caregiver at CypressCaregiver at Cypress
Kristin WaldroupKristin Waldroup
Waitress at CypressWaitress at Cypress