Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

RAHUL REDDY

New York,NY

Summary

Results-driven Data Engineer with expertise in optimizing real-time data pipelines at Travelers Insurance, achieving a 40% reduction in processing latency. Proficient in AWS and Spark, I excel in enhancing data observability and driving operational efficiency. Strong collaborator with a focus on delivering impactful analytics solutions.

Overview

2
2
years of professional experience
1
1
Certification

Work History

Data Engineer

Travelers Insurance
New York, NY
01.2023 - Current
Real-Time Streaming Pipeline for CAT Insurance Data
  • Reduced processing latency by 40%, optimizing Kafka, Flink & AWS Kinesis for real-time data ingestion.
  • Enabled real-time anomaly detection, improving event-driven alerting by 7 bps with AWS Lambda & Kinesis.
  • Scaled system capacity, handling 4x data growth with Kafka partitioning & Flink checkpointing.
  • Developed interactive monitoring dashboards, boosting 8% engagement with Python Dash.
  • Minimized system downtime, automating infrastructure provisioning to improve deployment speed by 2x.
  • Accelerated CI/CD deployment by 70%, reducing manual overhead with Terraform & Jenkins.
Scalable Data Acquisition & Storage Framework
  • Cut ETL execution time in half, optimizing Databricks, PySpark & Spark SQL workflows.
  • Automated pipeline orchestration, lowering manual intervention by 30% with Apache Airflow.
  • Enhanced data observability, improving anomaly detection accuracy by 7 bps via ELK Stack & CloudWatch.
  • Boosted query performance by 50%, accelerating data retrieval in Snowflake for real-time analytics.
  • Saved $10K annually in cloud costs, optimizing AWS infrastructure with Terraform.
  • Improved data accuracy, leading to 17% expansion in data-driven user insights.
  • Strengthened business intelligence workflows, driving a 15% increase in operational efficiency.
Data Engineer | Amazon India | Aug 2018 – Nov 2021
  • Optimized ETL pipelines, reducing data processing time by 40% for Prime Sales Core.
  • Enhanced campaign performance, achieving a 7% lift in conversion rates for Prime product sales.
  • Developed an automated KPI onboarding system, reducing setup time by 5x with a Scala-based framework.
  • Scaled data infrastructure, processing 3x more data efficiently with PySpark & Hive.
  • Improved query execution speed by 15%, optimizing SQL & Snowflake for large-scale analytics.
  • Implemented real-time streaming, lowering event processing latency by 7 bps with Spark & HBase.
  • Reduced AWS expenses, cutting storage and compute costs by $10K through S3 & Glue optimizations.
  • Increased ETL reliability, reducing system failures by 0.07% with Apache Airflow automation.

Data Visualization | Storytelling with Data

● Presented insights to leadership through data-driven storytelling, translating complex dashboard analysis to action items.

● Provided data support to Marketing, Biz Ops & Finance team building dashboards for compliance, audits, and accounting.

Data Engineering | Ensuring High Data Quality

● Developed marketing foundational data model and designed UTM schema, enabling scalable and quick analysis.

● Resolved discrepancies between backend attribution data and amplitude, decreasing CAC by 14% and data reliability.

KPI Development & Measurement | Driving Business Impact

● Built a comprehensive dashboard for weekly business reviews, featuring key alerting metrics on product adoption, engagement, churn, retention, on-time payments, cure rates, and user resurrection, providing a holistic business overview.

Education

Masters - Computers and Information Science

Southern Arkansas University
Magnolia, AR

B-Tech - Electronics and Communication Eng.

Jawaharlal Nehru Technological Uni
India

Skills

Technical: Java, Python, Scala, SQL, (MySQL, Hive, Redshift, MongoDB, Cosmos DB), Spark, Flink, Databricks, HDFS, MapReduce, Kafka, REST API

Cloud & DevOps: AWS (Lambda, Redshift, S3, RDS, CloudWatch, EC2, Terraform), Azure (Synapse Analytics, ADF, Blob Storage, Azure DevOps, Cosmos DB), Apache Airflow, Kubernetes, Kibana, Jenkins, Maven, GitHub, Linux, CI/CD

Data Engineering & Analytics: Snowflake, Power BI, DBT, SSIS, Machine Learning, AI/ML, SAS, Data Warehousing, ETL, Unit Testing, Data Pipelines, Data Lake, Data Modeling, Data Quality & Observability

Certification

AWS Certified Data Analytics - Specialty

https://drive.google.com/file/d/1xwpfN6D3ECk6GGAg_bgZmVSaewBpqQ63/view

Timeline

Data Engineer

Travelers Insurance
01.2023 - Current

Masters - Computers and Information Science

Southern Arkansas University

B-Tech - Electronics and Communication Eng.

Jawaharlal Nehru Technological Uni
RAHUL REDDY