Mitali Singh
Dallas,TX
Summary
Accomplished Data Engineering professional with 15 years of experience designing and optimizing data platforms across Ecommerce, Financial Services, and Retail. Expertise in Snowflake, AWS, and Databricks enables the delivery of scalable solutions for high-volume data processing. Skilled in pipeline orchestration and architecture design, leveraging Apache Airflow to enhance efficiency and support enterprise-wide analytics.
Overview
15
15
years of professional experience
1
1
Certification
Work History
VP, Data Engineering Team Lead
Texas Capital Bank
06.2023 - Current
- Designed and implemented data pipelines using AWS Glue, Lambda, MWAA, and Snowflake for large-scale warehousing.
- Led orchestration and architectural design of data pipelines to modernize enterprise data infrastructure, enhancing scalability and performance.
- Developed Python-based solutions for ingestion, transformation, and reporting of high-volume datasets.
- Streamlined CI/CD deployments using GitHub, Azure DevOps, and Azure Pipelines, increasing reliability of data engineering solutions.
- Enhanced system reliability by implementing monitoring and observability with CloudWatch and Datadog.
- Collaborated with Data Governance team to integrate Collibra for metadata management and lineage tracking.
- Translated business requirements into scalable technical solutions, facilitating efficiency and advanced analytics.
- Mentored engineers and established best practices in data engineering and governance, fostering a culture of continuous improvement.
Lead Data Engineer
Cognizant Technologies | CapitalOne
Dallas, Texas
11.2022 - 05.2023
- Led integration of diverse data sources into cohesive analytical frameworks.
- Optimized ETL processes to enhance efficiency and reliability in data flows.
- Monitored job execution logs to ensure optimal performance of ETL jobs in production.
- Designed data pipelines for scalable processing and analytics.
- Evaluated tools and technologies to enhance data engineering capabilities.
- Defined metrics and identified areas for improvement in data engineering tasks.
Lead Data Engineer
Tata Consultancy Services Ltd | Shutterfly LLC
10.2019 - 10.2022
- Designed and implemented distributed data processing pipelines using Apache Spark, Hive, and Python.
- Optimized Spark jobs and Redshift queries, resulting in significant performance improvements.
- Implemented Delta Lake features to resolve backfill and re-ingestion challenges.
- Created Databricks notebooks for transforming and loading data into Snowflake, improving data accessibility.
- Orchestrated complex workflows using Airflow DAGs, streamlining data processing tasks.
- Managed metadata and lineage in AWS Data Lake to ensure visibility and quality.
- Leveraged AWS services including EC2, S3, and CloudWatch for efficient operations.
- Collaborated with team members using GitHub for version control and project management.
Sr. Data Engineer
Tata Consultancy Services Ltd | Shutterfly LLC
01.2017 - 10.2019
- Migrated on-premise data warehouse to AWS Redshift using DMS and Schema Conversion Tool, enhancing data accessibility.
- Designed scalable data pipeline frameworks for automating ingestion, processing, and delivery of structured and unstructured data.
- Automated data ingestion processes with PySpark and Python, streamlining workflows and reducing manual intervention.
- Constructed AWS pipelines incorporating VPC, EC2, S3, Auto Scaling Groups, Snowflake, IAM, CloudWatch, and CloudTrail.
ETL Developer
Tata Consultancy Services Ltd | AMEX
08.2014 - 01.2017
- Created ETL graphs to ensure smooth HDFS integration.
- Developed and enhanced Data Warehouse applications using Ab Initio.
- Managed end-to-end delivery and testing strategies while providing UAT support.
- Utilized Ab Initio, Oracle, Teradata, Control-M, Linux, JIRA, and Agile methodologies.
Programmer Analyst
Multiple Clients ( TCS Bancs Securities Processing In house Product)
01.2011 - 08.2014
- Leveraged tech stack, including Java, Oracle, Linux, JIRA, Agile, and Git, to streamline development processes.
- Utilized Java-based applications to support various project requirements.
- Executed SQL tuning and Unix/Linux scripting, resulting in improved application performance.
- Built SWIFT interfaces and proprietary messaging solutions to facilitate seamless communication.
- Developed modules for trade settlements and position management within capital markets.
Education
Master's - Computer Application
Indra Gandhi Institute of Technology
India01.2009
Skills
- Programming languages: Python, Java
- Data processing: PySpark, Spark SQL
- Database management: SQL, PLSQL
- Scripting: Shell scripting
- Cloud platforms: AWS, Snowflake
- Data orchestration: Databricks, Apache Airflow
- ETL tools: Ab Initio, Informatica
- Job scheduling: Control-M, Automic
- Version control: GitHub, JIRA
- Development environments: Pycharm, Teradata
- Database systems: Oracle, Redshift
- Agile methodologies: Scrum
- Data governance strategies
- Operating systems: Unix/Linux
Certification
- AWS Certified Solutions Architect - Associate, 2025-2028
- AWS Developer - Associate
- Databricks Certified Data Engineer Associate
- Databricks Academy Accreditation - Generative AI Fundamentals, 2024-2026
- Databricks Associate Developer - Spark
- Python Certified Entry-Level Programmer (PCEP)
- Astronomer Certification - Apache Airflow Fundamentals & DAG Authoring
- Big Data Analytics with Hadoop and Apache Spark (LinkedIn)
- Data Engineering Foundations (LinkedIn)
Timeline
VP, Data Engineering Team Lead
Texas Capital Bank
06.2023 - Current
Lead Data Engineer
Cognizant Technologies | CapitalOne
11.2022 - 05.2023
Lead Data Engineer
Tata Consultancy Services Ltd | Shutterfly LLC
10.2019 - 10.2022
Sr. Data Engineer
Tata Consultancy Services Ltd | Shutterfly LLC
01.2017 - 10.2019
ETL Developer
Tata Consultancy Services Ltd | AMEX
08.2014 - 01.2017
Programmer Analyst
Multiple Clients ( TCS Bancs Securities Processing In house Product)
01.2011 - 08.2014
Master's - Computer Application
Indra Gandhi Institute of Technology