Vishal Sharma
Frisco
Summary
Data Engineer adept at creating scalable data architectures and optimizing workflows across AWS and Snowflake environments. Developed complex ETL pipelines and real-time data streaming solutions, collaborating with cross-functional teams to translate business needs into technical solutions. Strong focus on enhancing data quality and compliance through robust governance practices.
Overview
19
19
years of professional experience
Work History
Senior Data Engineer
DTCC/ITech US
Coppell
08.2021 - Current
- Developed and optimized complex SQL queries, views, and stored procedures, significantly improving execution time and reliability.
- Implemented data governance practices to ensure compliance and data quality standards.
- Supported data warehouse modeling using star and snowflake schemas to enable efficient BI reporting.
- Automated and scheduled large-scale batch workflows using AutoSys, ensuring reliable job execution and SLA compliance.
- Utilized Databricks to enhance banking operations and perform infrastructure analytics.
- Built scalable data lake architecture on AWS S3 using bronze/silver/gold data layers.
- Developed high-performance ETL/ELT pipelines using Python, PySpark, and Apache Spark on Databricks.
- Implemented Snowflake data warehouse solutions for financial reporting, risk analytics, and regulatory compliance.
- Built real-time streaming pipelines using AWS Kinesis and Kafka for transaction monitoring and infrastructure event processing.
- Developed AI/ML-ready data pipelines enabling fraud detection and credit risk modeling use cases.
- Automated infrastructure provisioning with Terraform, streamlining deployment processes across AWS environments.
- Orchestrated workflows using Apache Airflow for scheduling and dependency management.
- Improved data processing performance by 50% through Spark optimization and partitioning strategies.
- Collaborated with analysts and stakeholders to translate business requirements into effective technical solutions.
- Monitored and tuned database performance, proactively resolving bottlenecks and failures.
- Tested lower environments and production to ensure deployment of business rules.
- Worked closely with stakeholders to optimize BI reports, ensuring they aligned with key business goals and improved the accessibility and reliability of data for end users.
- Prioritize JIRAs for enhancements or reported issues for current functionality for upcoming sprints.
Data Engineer
Wells Fargo/ITech US
Charlotte
06.2019 - 07.2021
- Developed scalable ETL pipelines using AWS Glue, Lambda, S3, and Redshift for banking transaction processing.
- Developed data pipelines in Snowflake to facilitate seamless data flow across systems.
- Built and optimized Databricks Spark pipelines for large-scale financial datasets.
- Migrated legacy on-prem data warehouse systems to Snowflake cloud data platform.
- Gathered business and technical requirements, conducted source system analysis and gap analysis, and documented ETL development and reporting requirements.
- Designed data models for enterprise reporting and regulatory compliance dashboards.
- Developed and maintained complex SQL queries to extract key business insights.
- Developed Python automation scripts for reconciliation, validation, and data quality checks.
- Collaborated with data science teams to support fraud detection and risk analytics pipelines.
- Analyzed database performance and identified optimization opportunities for improved efficiency.
- Reduced data latency and improved query performance by 35–40%.
Database Analyst
Wells Fargo/Signature Consulting
Charlotte
03.2016 - 09.2018
Database Analyst
Fuze
San Francisco
02.2015 - 03.2016
Sr. Business/Database Analyst
Wells Fargo
San Francisco
10.2013 - 02.2015
Business/QA Analyst
Cetera Financial
El Segundo
04.2013 - 10.2013
Business/QA Analyst
Wells Fargo
Minneapolis
10.2011 - 03.2013
QA Analyst
Tata Indicom
India
08.2007 - 12.2008
Education
Master of Science - Electrical Engineering
SJSU
Bachelor of Electronics & Comm. -
PTU
Punjab, INDIASkills
- ETL pipelines
- Data warehousing
- Lake architecture
- Data modeling
- Feature engineering
- ML pipelines
- AI datasets
- Apache spark
- Pyspark
- Spark streaming
- Databricks
- Kafka
- Kinesis
- Redshift
- Snowflake
- Aws
- S3
- Glue
- Lambda
- Ec2
- Terraform
- Docker
- Ci/cd
- Jenkins
- Github actions
- Python
- Sql
- Mysql
- Tableau
- Power bi
- Airflow management
Locations
- Coppell, Coppell, TX, 75019
- Charlotte, Charlotte, NC, 28202
- San Francisco, San Francisco, CA, 94105
- El Segundo, El Segundo, CA, 90245
- Minneapolis, Minneapolis, MN, 55401
- India, India
Core Skills
python, sql, pyspark, aws (s3, glue, lambda, ec2, iam, kinesis, redshift), snowflake, databricks, apache spark, kafka, aws kinesis, spark streaming, etl/elt pipelines, data warehousing, data lake architecture, data modeling, feature engineering, ml data pipelines, ai-ready datasets, apache airflow, terraform, docker, ci/cd (jenkins/github actions), postgresql, oracle, mysql, data quality, data lineage, metadata management, tableau, power bi
Timeline
Senior Data Engineer
DTCC/ITech US
08.2021 - Current
Data Engineer
Wells Fargo/ITech US
06.2019 - 07.2021
Database Analyst
Wells Fargo/Signature Consulting
03.2016 - 09.2018
Database Analyst
Fuze
02.2015 - 03.2016
Sr. Business/Database Analyst
Wells Fargo
10.2013 - 02.2015
Business/QA Analyst
Cetera Financial
04.2013 - 10.2013
Business/QA Analyst
Wells Fargo
10.2011 - 03.2013
QA Analyst
Tata Indicom
08.2007 - 12.2008
Master of Science - Electrical Engineering
SJSU
Bachelor of Electronics & Comm. -
PTU