Anil Hardageri
Pflugerville,TX
Summary
Highly accomplished and results-driven Lead Data Engineer and Architect with 12+ years of comprehensive experience architecting and operating mission-critical, enterprise-scale data platforms for high-velocity environments (Amazon, BMO Financial). Expert in defining technical roadmaps, translating business needs into highly resilient data solutions, and ensuring financial accuracy. Proven success mentoring and leading high-performing teams and driving major Infrastructure-as-Code (IaC) initiatives. Pioneer in building scalable Data Lakehouses leveraging Apache Iceberg and zero-touch automation via real-time Spark and event-driven architecture.
Overview
7
7
years of professional experience
Work History
Lead Data Engineer
Amazon.com
Austin, TX
07.2021 - Current
- Strategy & Leadership: Served as the technical team lead, mentoring and supervising a combined team of 10 Data Engineers, Data Scientists, and Business Intelligence professionals.
- Data Modeling & Optimization: Designed and implemented optimized data models using a Star Schema approach within the security domain, which were utilized by cross-domain teams. This initiative significantly improved query performance and reduced the time for joining tables and writing analytical queries by over 6 hours, leading to more efficient dashboarding/reporting work.
- Architecture & Reliability: Drove Infrastructure-as-Code (IaC) initiatives using AWS CDK (TypeScript/Python), implementing automated CI/CD pipelines that reduced deployment time by 60%.
- Architected and implemented an Enterprise-scale Data Lakehouse using Apache Iceberg table format on S3, enabling ACID transactions and time travel, resulting in 30% reduction in data processing costs.
- Engineered a near real-time monitoring system for global transportation using event-driven architecture (SNS/SQS/Lambda), detecting shipping anomalies within 5-minute SLA.
- Architected near real-time Spark Job by transforming complex data type columns, reducing data latency to under 10 minutes and saving the company $1MM annually.
- Built a Text to SQL conversion application using LLMs and RAG as knowledge store, leveraging AWS Bedrock and OpenSearch for schema-aware query generation. Implemented automated query execution with Athena as the execution engine, reducing ad-hoc data requests by 40%, and enabling self-service analytics for business users.
- Spearheaded Infrastructure-as-Code initiatives using AWS CDK (TypeScript/Python), implementing automated CI/CD pipelines that reduced deployment time by 60% and achieved 100% infrastructure compliance through reusable, tested components for Glue, Step Functions, and security configurations.
Senior Data Engineer
BMO Financial Corporation
Chicago, IL
07.2020 - 07.2021
- Financial Domain Exposure: Developed and maintained ETLs to create visualization and Business Review (BR) reports.
- Platform Migration: Migrated legacy SQL Scripts and re-engineered them to run on the Spark ecosystem using AWS GLUE, which improved the total run time by 50%.
- Developed code modularization framework end to end for data science/machine learning modeling using Python.
Data Scientist- Fleet Planning
Hertz Corporation
Estero, FL
11.2018 - 07.2020
- Built Linear Mixed Effects model to predict mileages of rental fleet by car class, pool, and area level based on historical rentals.
- Built a Python scrapper to get pricing and inventory data of competitors on daily cadence.
Education
Master of Science - Industrial Engineering
University of Houston
Houston, TX05.2016
Bachelor of Engineering - Mechanical Engineering
Visvesvaraya Technological University
Belgaum, India06.2012
Skills
Data Platforms & ModelingApache Iceberg, Spark Ecosystem (Glue, Databricks), Data Lakehouse Architecture, Star Schema, S3, Redshift, DynamoDB, OpenSearch, ETL/ELT, Hive
Cloud Architecture & IaCAWS CDK (TypeScript/Python), Infrastructure-as-Code (IaC), CloudFormation, Lambda, StepFunctions, Athena, Glue
Automation & Real-TimeEvent-Driven Architecture (SNS/SQS), Automated CI/CD Pipelines, AWSCodePipeline, Git, CodeCommit
Programming & AnalyticsPython, SQL, TypeScript, PartiQL, Tableau, QuickSight, VS Code
Timeline
Lead Data Engineer
Amazon.com
07.2021 - Current
Senior Data Engineer
BMO Financial Corporation
07.2020 - 07.2021
Data Scientist- Fleet Planning
Hertz Corporation
11.2018 - 07.2020
Master of Science - Industrial Engineering
University of Houston
Bachelor of Engineering - Mechanical Engineering
Visvesvaraya Technological University