VIPINKUMAR MARAMRAJ
Atlanta,GA
Summary
Professional Data Engineer with around 7 years of industry experience in Python development, data analytics, building pipelines, working with various data sources, tools and pulling insights from data. Outstanding communication skills, dedicated to maintain up-to-date IT skills and industry knowledge.
Overview
7
7
years of professional experience
Work History
Data Engineer
HelloFresh
01.2022 - Current
- Designed and implemented end-to-end ETL data pipelines to extract, transform, and load data from various sources into the data warehouse using Python and SQL.
- Enhanced data quality by performing thorough cleaning, validation, and transformation tasks.
- .Fine-tuned query performance and optimized database structures for faster, more accurate data retrieval and reporting.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
- Implemented Snowflake Pipe, Task and Stream to ingest streaming data from Amazon S3 into Snowflake data warehouse, enabling real-time analysis of large-scale datasets and supporting data-driven decision-making processes
- Designed and implemented real-time data processing pipelines using Kafka streaming, enabling the ingestion, processing, and analysis of high-volume data streams.
- Orchestrated containerized applications using Amazon EKS , leveraging Kubernetes for container orchestration and management in the AWS cloud environment.
- Implemented data ingestion pipelines using Kafka Connect , leveraging connectors for integrating with various data sources and sinks such as databases, file systems, and cloud services.
- Implemented data partitioning and indexing strategies to optimize query performance on large datasets. Created complex and robust Airflow DAGs to orchestrate ETL workflows , ensuring data pipelines run efficiently and reliably.
- Utilized version control systems (e.g., Git ) to manage ETL code and ensure smooth collaboration with the data engineering team.
- Implemented CI / CD pipelines for automated deployment of ETL code changes, promoting continuous integration and delivery practices.
- Maintained comprehensive technical documentation for ETL processes and Airflow DAGs.
Data Engineer / Contractor
BCG / Client: UPS
08.2018 - 01.2022
- Worked on building a Forecasting Model which estimates weekly and monthly volumes using Prophet on a 12-18 months horizon.
- Worked on feature selection using factor analysis to identify the required categories from more than 1000 merchants online and B&M transactions data, which are needed to model the volumes.
- Worked with external vendors in identifying external data which is needed in model to estimate market size.
- Implemented ETL jobs and streaming processes required to ingest and transform data in cloud environment.
- Utilized Multiprocessing in Python to optimize the code running time and improve the processing speed, which increased code efficiency by 50%.
- Created tables and views in SQL Server to move data from flat files using Pandas and SQL Alchemy.
Data Engineer / Contractor
BCG / Client: AT&T
03.2018 - 08.2018
- Worked in designing data ingestion pipeline for Packet Optical Network team to process and ingest network data from different sources using TICK Stack, Kafka and stored processed data in InfluxDB.
- Developed scripts for stream processing and batch processing of data and wrote scripts to generate email notifications.
- Created and maintained various dashboards in Grafana which shows various network statistics.
- Utilized Kafka to collect metrics from collector agents and send them to InftuxDB , which enables to connect to multiple collector agents.
Software Engineer
BCG
09.2017 - 03.2018
- Developed web application and designed Forms, Models, Views and templates using Python and Django framework, respectively.
- Worked on object-oriented programming (OOP) concepts using Python, Django and Linux.
- Implemented RESTful web-services for sending and receiving the data from multiple systems.
- Developed SQL stored procedures for data manipulation and querying data from SQL Server
Education
Master's in Computer Engineering -
New York Institute of Technology
Old Westbury, NY07.2017
Bachelor's in Electronics and Communication Engineering -
Kakatiya University
India05.2015
Skills
- Python
- SQL
- SQL Server
- PostgreSQL
- AWS
- Snowflake
- Big Query
- Kafka
- Docker
- Airflow
- ETL development
- Data Modeling
- Data Pipeline Design
Timeline
Data Engineer
HelloFresh
01.2022 - Current
Data Engineer / Contractor
BCG / Client: UPS
08.2018 - 01.2022
Data Engineer / Contractor
BCG / Client: AT&T
03.2018 - 08.2018
Software Engineer
BCG
09.2017 - 03.2018
Master's in Computer Engineering -
New York Institute of Technology
Bachelor's in Electronics and Communication Engineering -
Kakatiya University
Similar Profiles
Chirag JainChirag Jain
Associate Director of Transportation Strategy at HelloFreshAssociate Director of Transportation Strategy at HelloFresh
ANDREW COUNCILANDREW COUNCIL
Production Forklift Operator at HellofreshProduction Forklift Operator at Hellofresh
Austin WrightAustin Wright
Human Resources Associate at HelloFreshHuman Resources Associate at HelloFresh