Senior BI/Data Engineer
Amazon
- Migrated 10k+ datasets across 35 Redshift databases from legacy systems to modern big-data technologies with Amazon's fine-grain control, enhancing performance and scalability while ensuring minimal business impact..
- Engineered end-to-end data pipelines leveraging AWS services (DynamoDB, Glue, S3, Lambda) and internal ETL tools to deliver actionable MBR/WBR insights, enabling rapid business decision-making.
- Successfully implemented design changes and access management protocols, reducing both CPU utilization and disk space usage from 80% to 50% across the data warehouse fleet.
- Developed Python automation scripts to optimize operations and user limits, significantly reducing long-running jobs, error incidents, and on-call support requirements.
- Designed and implemented 15 Quicksight and 10 Tableau interactive dashboards with user-friendly visualizations to provide real-time insights into key safety and compliance metrics.
- Architected end-to-end CI/CD deployment using BDT Maestro for ETL pipelines and implemented Airflow orchestrator for automated workflow monitoring, ensuring reliable pipeline execution and timely alerting.
- Fine-tuned query performance and optimized database functionality for faster, more accurate data retrieval and reporting to meet sev 2 SLA's
