Sonam Jain
Mountain View,CA
Summary
Senior Data Engineer with 10+ years of experience building scalable data platforms, real-time analytics systems, and AI-powered applications. Proven track record of leading end-to-end development of complex data and AI solutions from architecture to product driving measurable business impact.
Specialized in LLM-powered systems, agentic workflows, and data-driven decision platforms, with expertise in distributed data processing, cloud architectures, and advanced analytics. Recognized for influencing cross-functional stakeholders, mentoring teams, and delivering high-impact solutions that improve efficiency, reliability, and insight generation.
Passionate about building trustworthy, scalable AI-enabled data ecosystems that empower organizations with faster, smarter decision-making. Insightful Senior Data Engineer known for high productivity and efficiency in task completion. Possess specialized skills in data modeling, ETL development, and cloud computing solutions. Excel in problem-solving, teamwork, and communication, ensuring successful project outcomes and effective collaboration with cross-functional teams.
Overview
16
16
years of professional experience
Work History
Senior Data Engineer
Amazon (Music | Prime Video)
Sunnyvale, CA
12.2019 - Current
- Built a full-stack, multi-agent AI analytics assistant with LLM reasoning, tool orchestration, and knowledge base routing, enabling natural language-driven analytics and automating WBR reporting, reducing manual analysis effort by 40–50% and turnaround time by 60–70%
- Designed and implemented a Ground Truth evaluation framework using SQL-based recipes and parameterized execution, establishing a single source of truth for analytics validation and enabling reproducible benchmarking
- Designed and launched AI-powered anomaly detection platform processing 1B events/day, enabling resolution of 100 critical issues/month by executives and incorporating customer feedback for actionable insights.
- Led engineering design and data integration for Maestro (curated playlists) and expansion of Amazon Music to include Audiobooks and Podcasts, delivering engagement reporting that enabled leadership to track adoption and optimize features post-launch.
- Led the development of the User Journey Platform (UJP), implementing sessionization, playback, and attribution frameworks to power widget-level analytics, content recommendation, and engagement metrics across 8+ domains (STOs) for WBR/MBR reporting, transforming billions of events into query-ready datasets and reducing analysis time by 70%.
- Designed a governed data ingestion framework for 3P partner data, establishing schema standardization, validation, and access controls to enable secure, scalable, and trusted analytics
Senior Data Scientist
Microsoft (Microsoft Teams)
Sunnyvale, CA
01.2017 - 11.2019
- Built an end-to-end data quality and validation framework processing 25M+ signals/hour, generating high-confidence metrics to ensure correctness, completeness, and low-latency delivery for real-time decision-making
- Developed statistical models (time series, regression) to identify correlations and causality between KPIs and service performance, accelerating root cause analysis and improving incident resolution
- Designed and implemented a service health analytics platform, enabling proactive detection of customer-impacting issues and driving improved reliability and operational efficiency across partner teams
- Defined and operationalized 30+ service health KPIs and real-time monitoring dashboards, improving observability and reducing mean time to detect (MTTD) and resolve (MTTR)
Senior Data Engineer
Citrix Systems Inc
San Francisco Bay Area
12.2014 - 01.2017
- Built a recommendation engine for sales lead ranking and upsell, improving qualified lead conversion by 7%
- Developed Customer 360 MDM solutions using fuzzy matching and deduplication, reducing duplicate records by 90% and improving data quality
- Architected a cloud-based AWS data platform consolidating siloed datasets into a unified analytics environment, reducing data delivery time from 4 hours to and enabling near real-time reporting
Data Engineer
Tata Consultancy Services
Greater New York City Area & India
12.2009 - 12.2014
- Architected and implemented ETL frameworks integrating data from 15+ heterogeneous systems (ERP, CRM, SQL/NoSQL, flat files) into centralized data warehouses
- Led Oracle ERP R12 rollouts across APAC (Singapore, Malaysia, China, Taiwan), delivering multiple modules for 1,000+ enterprise users
- Developed high-performance PL/SQL ETL pipelines with parallel processing, reducing batch runtimes from 8 hours to
- Built data quality and audit automation frameworks, validating 100% of records and eliminating recurring reconciliation errors
- Delivered large-scale data migrations (500K+ records) using Oracle APIs with zero downtime
- Designed SCM performance dashboards, reducing procurement cycle time by 15%
Education
Bachelor's Degree - Computer Science
Rajiv Gandhi Institute of Technology
12-2009
Skills
- Multi-agent systems
- LLM applications and orchestration (Lang Graph Strands)
- Retrieval-augmented generation
- Knowledge base
- Prompt engineering
- LLM evaluation
- Distributed data processing (Apache Spark, PySpark)
- Advanced SQL, Data modeling
- ETL/ELT pipelines, Real-time & batch processing
- Data quality, validation frameworks, and data governance
- AWS (Lambda, S3, Redshift, Athena, DynamoDB, Step Functions, EMR)
- Event-driven architecture, Scalable system design
- Python, SQL, Scala, React
- Tableau, QuickSight
Timeline
Senior Data Engineer
Amazon (Music | Prime Video)
12.2019 - Current
Senior Data Scientist
Microsoft (Microsoft Teams)
01.2017 - 11.2019
Senior Data Engineer
Citrix Systems Inc
12.2014 - 01.2017
Data Engineer
Tata Consultancy Services
12.2009 - 12.2014
Bachelor's Degree - Computer Science
Rajiv Gandhi Institute of Technology