Experienced Data Engineer with 4+ years of expertise in Data Analysis and Data Engineering, including working with large datasets (structured and unstructured), data acquisition, data validation, predictive modeling, and data visualization. Proficient in designing and deploying business intelligence, analytics, and reporting solutions using advanced data analysis techniques and software development skills. Adept at using databases, programming languages, and business intelligence platforms to build scalable data pipelines and actionable dashboards.
1. Project: Cloud Data Migration & Analysis for Dealer Data Transition.
Description:
Worked the end-to-end dealer data migration from the SHAW system to the AWS-based POTF platform, utilizing Snowflake for data warehousing and analysis. Built and automated scalable ETL processes using Python, Arrow, and AWS services to ensure seamless data ingestion. Developed advanced Tableau dashboards for visualizing migration progress and business insights. Worked closely with stakeholders to ensure technical solutions aligned with business objectives, improving data accessibility and insights.
Responsibilities:
Technologies & Tools: Python, AWS (S3, Lambda, EC2), Snowflake, SQL, Tableau, Arrow, GitHub, ETL Pipelines, Data Warehousing, Cloud Data Migration.
2. Project: Data Migration & Report Development for Snowflake and Tableau.
Description:
Worked on Snowflake for generating Tableau reports, collaborating closely with business stakeholders to gather requirements and define functionality. Developed performance reports that offered high-level overviews with the ability to drill down into specific details. Leveraged Python and AWS cloud services to automate processes and ensure efficient data management and report generation.
Responsibilities:
Environment: AWS (S3, Lambda, EC2), Python, Snowflake, Tableau, Power BI, SQL Server, MySQL, Oracle, Teradata, SSIS, Agile Methodologies, C#, JavaScript, MS Office, MS Visual Studio, SVN.
● Worked with team on developing and maintaining ETL (Extract, Transform, Load) pipelines using tools like Python, SQL to automate data extraction from various sources and load it into data storage systems.
● Worked on managing data in cloud-based data warehouses like Amazon Redshift, Snowflake ensuring the data is optimized for performance and accessibility.
● Lead meetings with business partners to define functional needs, which leads to the creation of system specifications and applications.
● Gained experience in cleaning and transforming raw data using tools like Pandas, PySpark ensuring consistency and quality across datasets.
● exposed to cloud services like AWS (S3, EC2, Lambda, RDS), Azure to set up scalable data pipelines and storage solutions in the cloud.
● Worked on creating basic data visualizations and reports using Tableau, Power BI.
● Gained Knowledge on version control systems like Git/GitHub to manage code, scripts, and pipeline configurations.
● Collaborate with cross-functional teams using tools like Jira, Confluence, and Slack to track tasks and projects, ensuring smooth communication and teamwork.
Environment: Windows 7, Linux, Tableau Desktop (10.x/2018.x/2019.x), Tableau Server (10.x/2018.x/2019.x), Microsoft SQL Server, Apache Hadoop, DB2, Informatica, Python, Java,Git, GitHub, JIRA, Agile,and Microsoft Excel.
Cloud Technologies:
AWS (S3, EC2, Lambda, Redshift), Azure
Data Warehousing & Databases
Snowflake, Amazon Redshift, Oracle, SQL Server
Programming & Scripting Languages
Python, SQL , Java/J2EE, R, SAS, Unix Shell Scripting (Bash)
ETL Tools
Python (for ETL automation), Informatica, Alteryx
BI &Reporting Tools:
Tableau (Server & Desktop), Power BI, Advanced MS Excel
Operating Systems
Windows, Linux (RHEL, CentOS)
Analytical Tools
R, SAS Enterprise Guide, SAS E-Miner, Adv MS Excel