Summary
Overview
Work History
Education
Skills
Certification
Additional Information
Timeline
Generic

Sivachandra Jampani

Irvine, CA,CA

Summary

Data Engineer with 5+ years of experience building scalable distributed data systems and custom ETL frameworks in high-volume cloud environments. Strong expertise in Python, PySpark, Spark, Hive, and Redshift, with hands-on experience designing data ingestion systems processing millions to billions of records weekly. Experienced in building API-driven extraction pipelines, optimizing distributed workloads, and collaborating closely with Data Scientists and Product teams in SaaS environments. Passionate about writing production-grade code and designing resilient, scalable data infrastructure.

Overview

5
5
years of professional experience
1
1
Certification

Work History

Data Engineer

Alliant Insurance
Irvine, CA
01.2024 - Current
  • Designed and deployed distributed ETL pipelines using Python and PySpark to process high-volume transactional datasets into Redshift and S3.
  • Built scalable ingestion systems integrating REST APIs, S3 event triggers, and batch processing jobs.
  • Migrated 3TB+ enterprise workloads to Amazon Redshift, optimizing schema design, distribution keys, and query performance.
  • Developed Spark-based transformations handling millions of records per batch with optimized partitioning and caching strategies.
  • Built monitoring and alerting systems using CloudWatch and logging frameworks to detect anomalies and infrastructure bottlenecks.
  • Implemented Airflow DAGs to orchestrate data extraction, transformation, and loading processes across distributed systems.
  • Partnered with Data Scientists to deliver curated datasets for modeling and advanced analytics use cases.

Data Engineer

Goldman Sachs
Hartford, CT
04.2023 - 12.2023
  • Built custom Python ETL pipelines ingesting structured and semi-structured data from APIs, RDBMS, and event streams.
  • Designed Spark-based distributed processing jobs on Azure and AWS environments.
  • Developed scalable data models for analytics using Redshift and Synapse SQL DW.
  • Improved ETL reliability by implementing automated validation checks and schema drift detection.
  • Automated infrastructure provisioning and deployments using Terraform and CI/CD pipelines.
  • Collaborated with engineering and product teams to design scalable data services supporting enterprise applications.

Associate Data Engineer

PharmEasy
Hyderabad, India
03.2021 - 12.2022
  • Developed high-throughput ingestion pipelines processing 1M+ records daily using Spark and Kafka.
  • Designed Hive-based transformations on Hadoop clusters for distributed batch processing.
  • Built Airflow DAGs to orchestrate multi-stage ETL workflows across distributed environments.
  • Implemented MongoDB and PostgreSQL integrations for application-level data storage.
  • Optimized Spark jobs using broadcast joins, partition pruning, and memory tuning techniques.

Education

Master of Science (M.S.) - Computer Science

Sacred Heart University
Fairfield, Connecticut
03-2024

Bachelor of Technology (B.Tech.) - Electronics and Communication Engineering

QIS College of Engineering and Technology
Ongole, India
07-2022

Skills

    Programming:
    Python (Advanced), PySpark, SQL, Scala, JavaScript (Nodejs), TypeScript (Working Knowledge)

    Big Data & Distributed Systems:
    Apache Spark, Hive, Hadoop, Databricks, Spark Streaming

    Databases & Storage:
    Redshift, PostgreSQL, MongoDB, Elasticsearch, S3

    ETL & Data Engineering:
    Custom Python ETL frameworks, API ingestion, Data validation pipelines

    Cloud Platforms:
    AWS (S3, Glue, Lambda, EMR, MSK, IAM), Azure

    Orchestration & Monitoring:
    Airflow, CloudWatch, Kibana

    DevOps:
    Terraform, Docker, CI/CD, Git

Certification

Databricks Certified Data Engineer Associate — Issued July 2025

Additional Information

Large-Scale Streaming Pipeline

Technologies: Spark, Kafka, Python, Redshift

  • Designed distributed streaming system processing 10M+ events daily.
  • Implemented real-time aggregation and anomaly detection logic.
  • Built fault-tolerant Spark jobs with checkpointing and replay capability.
Custom API Data Extraction Framework

Technologies: Python, REST APIs, S3, Redshift

  • Developed modular Python framework to extract, validate, and ingest data from multiple third-party APIs.
  • Implemented retry logic, rate-limit handling, and automated error recovery.
  • Reduced manual ingestion effort by 40%.

Timeline

Data Engineer

Alliant Insurance
01.2024 - Current

Data Engineer

Goldman Sachs
04.2023 - 12.2023

Associate Data Engineer

PharmEasy
03.2021 - 12.2022

Bachelor of Technology (B.Tech.) - Electronics and Communication Engineering

QIS College of Engineering and Technology

Master of Science (M.S.) - Computer Science

Sacred Heart University