Summary
Overview
Work History
Education
Skills
Timeline
Generic

Harshavardhan Mutra Rupendra

San Francisco,United States

Summary

I am a Data Engineer with 8+ years of experience designing scalable data platforms across retail, banking, and ed-tech domains. I specialize in Scala, Python, and SQL across distributed streaming and batch processing systems using Apache Flink, Airflow, BigQuery, SSIS, Teradata, and SQL Server. I build real-time analytics pipelines, optimize enterprise warehouses, develop dimensional and OLAP solutions, and implement governance frameworks that deliver high-performance, reliable, enterprise-grade data ecosystems.

  • Designing distributed streaming pipelines using Apache Flink, Pub/Sub, and BigQuery to enable real-time demand forecasting, clickstream enrichment, and behavioral analytics.
  • Constructing batch ETL frameworks using SSIS, Informatica, and SQL to consolidate ERP, POS, financial, and vendor datasets into curated warehouse layers.
  • Developing dimensional models, star schemas, and OLAP structures using SSAS and Teradata to support liquidity reporting, executive dashboards, and enterprise analytics.
  • Implementing data validation and governance controls using Great Expectations, Apache Ranger, IAM, and Collibra to enforce data quality, lineage, and secure access management.
  • Optimizing large-scale SQL and warehouse workloads through partitioning, indexing strategies, and execution plan tuning to improve throughput and reduce processing latency.
  • Engineering low-latency serving layers using Redis and ElasticSearch to support recommendation systems and dynamic pricing platforms.
  • Automating cloud-native deployments and CI/CD workflows using GitLab CI, Terraform, Docker, and Kubernetes to ensure reproducible infrastructure and reliable data releases.
  • Developing reconciliation logic and regulatory reporting processes using PL/SQL and T-SQL to strengthen financial audit controls and compliance transparency.
  • Implementing change data capture pipelines using Debezium and incremental loading patterns to synchronize operational systems with analytical warehouses.
  • Constructing feature stores using Feast and BigQuery to enable reusable machine learning features with training-serving consistency.
  • Building executive dashboards using SSRS and MicroStrategy to deliver KPI scorecards and operational performance insights.
  • Managing workflow orchestration using SQL Server Agent and Airflow DAGs to coordinate batch dependencies and ensure reliable pipeline execution.
  • Collaborating with cross-functional stakeholders to translate domain requirements into scalable transformations and governed reporting solutions aligned with Agile delivery practices.

Overview

8
8
years of professional experience

Work History

Senior Data Engineer

Amazon
United States
03.2024 - Current

Retail Demand Nexus — Built a real-time retail demand intelligence platform supporting inventory optimization and customer behavior analytics across global retail systems. Engineered distributed data pipelines and cloud-native data services enabling scalable streaming, ML feature generation, and high-performance analytics delivery.

  • Architected distributed streaming pipelines using Apache Flink and Google Pub/Sub, enabling near real-time clickstream enrichment and demand forecasting across global retail systems.
  • Built scalable ingestion frameworks using Apache NiFi and Cloud Composer, consolidating multi-source structured and semi-structured datasets into BigQuery analytical warehouses.
  • Implemented advanced stateful stream processing (windowing, watermarking) in Scala to improve inventory tracking accuracy and fulfillment visibility.
  • Designed graph-based product affinity models in Neo4j, enabling relationship analytics across customers, SKUs, and promotional campaigns.
  • Engineered low-latency serving layers using Redis and ElasticSearch, powering dynamic pricing engines and recommendation microservices.
  • Developed a centralized Feature Store using Feast and BigQuery, ensuring reusable ML features and training-serving consistency for personalization and elasticity modeling.
  • Implemented Change Data Capture pipelines (Debezium) and schema evolution strategies using Avro and Protobuf, ensuring data correctness and backward compatibility across streaming and batch systems.
  • Optimized BigQuery performance through partitioning, clustering, and cost-based query tuning, improving efficiency of large-scale retail analytics workloads.
  • Established data validation and secure access control frameworks using Great Expectations, Apache Ranger, IAM, and Data Catalog, enforcing quality, lineage, and governance compliance.
  • Automated cloud-native deployments using Terraform, GitLab CI, Docker, Helm, and Kubernetes, enabling reproducible infrastructure and scalable data services.
  • Built observability systems using Grafana and Prometheus, monitoring pipeline health, latency, and throughput across distributed clusters.

Environments: Scala, Apache Flink, Google Pub/Sub, Apache NiFi, Cloud Composer, BigQuery, Neo4j, Redis, ElasticSearch, Feast, Debezium, Avro, Protobuf, Apache Ranger, IAM, Data Catalog, Terraform, GitLab CI, Docker, Kubernetes, Helm, Grafana, Prometheus

Data Engineer

Goldman Sachs
United States
09.2022 - 02.2024

Capital Liquidity Vault — Built an enterprise-grade liquidity and regulatory reporting platform supporting Basel III compliance and treasury risk analytics. Engineered high-volume ETL pipelines and financial data models enabling accurate liquidity coverage and capital adequacy reporting across global banking operations.

  • Designed enterprise ETL frameworks using Informatica PowerCenter and Ab Initio, ingesting high-volume trade, settlement, and liquidity datasets into Teradata warehouses.
  • Built dimensional models (star and snowflake schemas) supporting Basel III liquidity coverage ratio and net stable funding ratio reporting.
  • Engineered PL/SQL and T-SQL reconciliation logic, strengthening financial validation, audit controls, and regulatory transparency across treasury systems.
  • Optimized Teradata performance through indexing strategies, partitioning, statistics collection, and execution plan analysis across large transactional datasets.
  • Implemented batch orchestration workflows using Control-M, ensuring dependency resolution and SLA adherence for time-sensitive regulatory submissions.
  • Integrated upstream market data feeds via IBM MQ, FTP, and SFTP, standardizing ingestion pipelines across treasury and enterprise risk systems.
  • Developed OLAP cubes in Microsoft SSAS and delivered executive dashboards using MicroStrategy and Cognos for liquidity, capital adequacy, and funding gap analysis.
  • Strengthened data governance and lineage documentation using Collibra, improving metadata traceability and regulatory audit readiness.
  • Implemented data quality controls using Informatica Data Quality, enforcing completeness, consistency, and accuracy across financial datasets.
  • Tuned DB2 and Oracle databases to improve concurrent workload performance across reporting and reconciliation processes.
  • Designed audit logging mechanisms capturing validation checkpoints and exception handling to support compliance and internal audit reviews.
  • Supported disaster recovery validation and secure data transmission using encryption and SFTP protocols across financial integrations.

Environments: Informatica PowerCenter, Ab Initio, Teradata, Oracle, DB2, PL/SQL, T-SQL, Microsoft SSAS, MicroStrategy, Cognos, Collibra, Control-M, IBM MQ, UNIX

ETL Data Engineer

Kroger
United States
01.2020 - 08.2022

Retail Insights Hub — Developed a centralized retail analytics and reporting platform supporting merchandising, supply chain, and sales intelligence initiatives. Engineered scalable ETL pipelines and dimensional warehouse models to deliver operational KPIs and executive reporting across enterprise retail systems.

  • Designed scalable ETL workflows using SSIS and SQL Server, integrating point-of-sale, inventory, and supplier datasets into centralized retail data warehouses.
  • Built incremental data pipelines implementing change data capture patterns, audit logging, and error-handling logic for reliable batch processing.
  • Developed dimensional models and fact tables optimized for retail analytics across product hierarchies, fiscal calendars, and regional segments.
  • Engineered SSAS multidimensional cubes enabling sales, margin, and inventory performance analysis for merchandising and supply chain teams.
  • Delivered executive dashboards and operational KPI reports using SSRS, supporting revenue trend analysis and demand forecasting initiatives.
  • Optimized SSIS package performance through data flow tuning, buffer configuration, and parallel execution strategies to improve batch throughput.
  • Implemented slowly changing dimension handling using surrogate keys and merge operations within ETL workflows.
  • Automated job orchestration using SQL Server Agent, managing dependency chains and failure notifications across warehouse environments.
  • Integrated external vendor and pricing feeds into standardized warehouse schemas to support downstream retail reporting.
  • Managed production deployments of SSIS packages, SSAS cubes, and SSRS reports through structured release processes.

Environments: SQL Server, SSIS, SSAS, SSRS, T-SQL, SQL Server Agent, Data Warehousing, Star Schema, OLAP Cubes

Data Engineer

UpGrad
Mumbai, India
06.2018 - 12.2019

Learning Analytics Engine — Built a learning analytics platform supporting student engagement tracking and academic performance reporting. Engineered Python-based ETL pipelines and SQL warehouse models to deliver actionable insights across online education programs.

  • Designed relational schemas in MySQL to structure student enrollment, course progress, assessment, and certification datasets.
  • Developed Python-based ETL pipelines orchestrated through Apache Airflow, transforming raw activity logs into analytics-ready warehouse tables.
  • Built ingestion workflows processing CSV and JSON feeds, validating and cleansing data prior to structured storage.
  • Integrated third-party LMS APIs, mapping response payloads into standardized SQL staging tables for downstream reporting.
  • Developed stored procedures and automated grading logic to support learner performance tracking and academic reporting.
  • Designed aggregated data marts supporting retention analysis, cohort tracking, and longitudinal performance insights.
  • Optimized SQL queries to improve dashboard responsiveness and reporting efficiency in Tableau.
  • Managed structured synchronization between MongoDB collections and SQL analytical tables for unified reporting views.

Environments: Python, Apache Airflow, MySQL, MongoDB, Tableau, Linux

Education

Bachelor Of Engineering - Computer Science

BITS Pilani
07-2018

Skills

  • Programming Languages : Python, Scala, SQL, PL/SQL, T-SQL
  • Streaming & Big Data : Apache Flink, Pub/Sub, Debezium, BigQuery
  • ETL & Orchestration : SSIS, Informatica, Ab Initio, Apache Airflow, Control-M
  • Data Warehousing : Teradata, SQL Server, Oracle, DB2, Dimensional Modeling
  • Reporting & BI : SSRS, SSAS, MicroStrategy, Cognos, Tableau
  • Databases : MySQL, PostgreSQL, SQL Server, MongoDB
  • Cloud & DevOps : GCP, Terraform, GitLab CI, Docker, Kubernetes
  • Governance & Security : Apache Ranger, Collibra, IAM, Data Catalog
  • Monitoring & Performance : Grafana, Prometheus, JMeter, Query Tuning
  • Operating Systems & Tools : Linux, Windows Server, Shell Scripting, SVN, Git

Timeline

Senior Data Engineer

Amazon
03.2024 - Current

Data Engineer

Goldman Sachs
09.2022 - 02.2024

ETL Data Engineer

Kroger
01.2020 - 08.2022

Data Engineer

UpGrad
06.2018 - 12.2019

Bachelor Of Engineering - Computer Science

BITS Pilani
Harshavardhan Mutra Rupendra