Sneha Shetty
Dallas,TX
Summary
Data Engineer with 3 years of experience in designing and automating scalable data pipelines using SQL and Snowflake. Proficient in big data frameworks such as Apache Spark and Kafka, with a strong focus on CI/CD deployments and data modeling techniques. Delivered secure and cost-optimized data solutions for enterprise analytics and AI workloads across AWS, Azure, and GCP. Demonstrated ability to collaborate with cross-functional teams and translate complex technical requirements into actionable solutions.
Overview
5
5
years of professional experience
Work History
Data Engineer
BMO Financial Group
Chicago, IL
01.2024 - 04.2024
- Designed scalable ETL frameworks using AWS Glue, EMR, Step Functions, and Lambda, Python
- Built real-time data ingestion pipelines with Kafka, Kinesis, and Spark Streaming
- Managed Redshift + S3 Lakehouse architecture with data cataloging and partitioning
- Enabled CI/CD automation using Terraform, Code Pipeline, Docker, CloudFormation, streamlining deployment processes and reducing manual errors
- Delivered operational insights via Amazon Quick Sight dashboards, facilitating informed decision-making and strategic planning
- Developed Scalable Spark-based data pipelines in Azure Databricks for batch and real-time analytics, integrating with Delta Lake and Power BI
- Designed and orchestrated complex ETL workflows using Azure Data Factory, automating data movement across hybrid sources with dynamic mapping and trigger
- Proficient in data modeling techniques including Star Schema and Snowflake, enabling efficient querying and scalable analytics across OLTP and OLAP systems.
- Experienced data processing frameworks like Apache Spark, Kafka, and Airflow to build robust, distributed pipelines for batch and real-time workloads.
Data Engineer
Customer Value Partners (CVP)
Baltimore, MD
02.2022 - 12.2023
- Built shell/Python scripts, SSIS packages, and data warehouses in PostgreSQL and Teradata, enhancing data processing efficiency and storage capabilities
- Utilized Apache Spark, Hive, and NiFi to power large-scale ETL and reporting solutions, improving data processing speed and accuracy
- Applied advanced SQL, Redis caching, and data marts for analytics acceleration, resulting in faster query performance and improved data insights
- Developed serverless ETL workflows using Lambda, Python, and Snowflake, improving data processing efficiency and reducing operational costs
- Implemented robust data integration solutions using Informatica, enhancing data accuracy and streamlining data flow across systems
- Managed secure cloud infrastructure with VPC, subnets, and security groups
- Delivered large-scale transformations using Apache Spark, significantly improving data processing speed and scalability
- Experienced in leveraging Google Cloud Platform (GCP) tools such as Big Query, Dataflow for scalable data processing, analytics, and machine learning workflows. Skilled in optimizing ETL pipelines, cloud storage, and AI model deployment within GCP's ecosystem
Jr. Data Engineer
Abbott Labs
Chicago, IL
07.2021 - 01.2022
- Utilized AWS Glue and Synapse to streamline ETL and analytics workflows, enhancing data processing efficiency
- Created Python scripts and SQL queries for data transformation and validation, ensuring data accuracy and reliability
- Supported large datasets and optimized ingestion pipelines in RDS and Redshift, improving data accessibility and performance
Graduate Assistant (GA) Roles
University of Arkansas at Little Rock
Little Rock, AR
08.2019 - 05.2021
- Developed automated workflows in WordPress (Gravity Forms) for academic processes, improving efficiency and reducing manual errors
- Managed Jira, Bitbucket & Confluence access controls across research initiatives, ensuring secure and efficient collaboration
- Supported faculty and undergrad students in Python programming, enhancing their coding skills and project outcomes
- Participated in Agile research cycles, version control, and documentation workflows, contributing to streamlined project management and improved research outputs
Education
Master of Science - Computer Science & Technology
University of Arkansas at Little Rock
05.2021
Skills
- Airflow
- Amazon QuickSight
- Apache Spark
- AWS Code Pipeline
- AWS Glue
- AWS Lambda
- AWS EMR
- AWS Step Functions
- AWS S3
- Azure Data Factory
- Azure Synapse
- CI/CD
- DevOps
- Cloud Platforms
- Data Engineering
- Data Lakes
- SQL
- MY SQL
- Data Modeling
- Data Storage
- Data Privacy Compliance
- Docker
- DynamoDB
- GCP BigQuery
- Git
- Glue Catalog
- Governance
- Security
- Hive
- IAM
- Informatica
- Jenkins
- Kafka
- Kinesis
- KMS
- Kubernetes EKS
- Lake Formation
- MongoDB
- PostgreSQL
- Power BI
- Programming
- Scripts
- Python
- Pandas
- NumPy
- Boto3
- Redshift
- Shell Scripting
- Snowflake
- SQL Tuning
- SQL Optimization
- Tableau
- Talend
- Terraform
- Visualization
- VPC
Timeline
Data Engineer
BMO Financial Group
01.2024 - 04.2024
Data Engineer
Customer Value Partners (CVP)
02.2022 - 12.2023
Jr. Data Engineer
Abbott Labs
07.2021 - 01.2022
Graduate Assistant (GA) Roles
University of Arkansas at Little Rock
08.2019 - 05.2021
Master of Science - Computer Science & Technology
University of Arkansas at Little Rock
Similar Profiles
CHRISTOPHER MARSHALLCHRISTOPHER MARSHALL
SPECIALIST II at BMO FINANCIAL GROUP, BMOSPECIALIST II at BMO FINANCIAL GROUP, BMO
Jasmine BanisterJasmine Banister
Associate Banker/Teller at BMO Financial GroupAssociate Banker/Teller at BMO Financial Group
Kelsey HaightKelsey Haight
Tax Operations Specialist at BMO Financial GroupTax Operations Specialist at BMO Financial Group
Surya SahooSurya Sahoo
Lead Data Analyst at BMO Financial GroupLead Data Analyst at BMO Financial Group
Charmagne N. GelseyCharmagne N. Gelsey
Payroll Conversion Consultant at EmpowerPayroll Conversion Consultant at Empower