Experienced Data Engineer with over 4 years of expertise in designing and implementing data solutions using AWS, SQL, Snowflake, ETL processes, Hadoop, and Spark. Proficient in building scalable data pipelines, optimizing performance, and ensuring data integrity and security. Skilled in leveraging cloud technologies to drive data-driven insights and support business objectives. Adept at working in fast-paced environments and collaborating with cross-functional teams to deliver high-quality solutions. Strong problem-solving abilities and a commitment to continuous learning.
Overview
5
5
years of professional experience
Work History
Data Engineer
Northern Trust - SLN Systems
Chicago, IL
11.2022 - Current
Developed and maintained data pipelines to ingest, store, process and analyze large datasets in AWS S3 buckets.
Developed Spark applications on top of Hadoop clusters running on EMR for performing complex analytics operations.
Implemented automated monitoring of data flows using Cloudwatch and Lambda functions.
Performed maintenance tasks such as backups, restores, patching, capacity planning and performance tuning of databases on AWS EC2 instances.
Created ETL processes using Python scripts to move data from various sources into the target databases on AWS Redshift or RDS.
Oversaw migration of on-premises infrastructure to cloud platforms, ensuring minimal downtime.
Collaborated with development teams to integrate cloud services into software applications.
Developed and maintained CI/CD pipelines for seamless code deployment to cloud platforms.
Managed version control and deployment of data applications using Git, Docker, and Jenkins.
Implemented data visualization tools like Tableau and Power BI to create dashboards and reports for business stakeholders.
Data Engineer
Licious
Banglore
10.2019 - 12.2021
Developed Python scripts for extracting data from web services API's and loading into databases.
Streamlined data flow from diverse sources using ETL tools such as Talend, Informatica, and Airflow.
Collaborated with stakeholders across business units to define requirements for new BI initiatives leveraging AWS services like Athena, QuickSight and Glue.
Managed Data Lake architecture based on Apache Parquet files stored in S3 buckets and queried via Athena.
Automated cloud deployments using Terraform and Ansible to enhance operational efficiency.
Created SQL Server Integration Services packages for incremental and full loads of data from various sources.
Developed dashboards with Tableau to monitor key performance indicators.
Utilized advanced analytics tools such as SAS, SPSS, Excel PowerPivot, to manipulate large volumes of structured and unstructured data sets.
Implemented and optimized big data storage solutions, including Hadoop and NoSQL databases, to improve data accessibility and efficiency.
Education
Master of Science - Management Information Systems
Kent State University
Kent, OH
05-2023
BBA - Business Administration And Management
Jain University
Banglore
06-2021
Skills
Python Programming
SQL Querying
PowerBI Reporting
Apache Spark Mastery
Snow Flake
Amazon Web Services
Hadoop Expertise
ETL Design and Implementation
Projects
Research paper on comparison of Credit Risk Management between a public sector and private sector bank.
Crypto Currency Forecasting
Anomaly detection in Surveillance videos
Timeline
Data Engineer
Northern Trust - SLN Systems
11.2022 - Current
Data Engineer
Licious
10.2019 - 12.2021
Master of Science - Management Information Systems
Kent State University
BBA - Business Administration And Management
Jain University
Similar Profiles
Pooja ChandakPooja Chandak
Associate Consultant -Hedge Funds at Northern TrustAssociate Consultant -Hedge Funds at Northern Trust