Priyanka Shelar
Somerset
Summary
Lead Data Engineer | Solutions Engineering | Cloud Data Platforms | Data Pipelines | Customer-Focused Solutions
Innovative technology professional with 14 years of diverse experience. Skilled in enhancing systems and aligning technical solutions with business objectives. Proven success in leading projects from start to finish and contributing to organizational growth and success.
Overview
16
16
years of professional experience
Work History
Lead Data Engineer
Analytic Partners
01.2023 - Current
- Led a team of engineers to design, develop, and deliver scalable data pipelines and analytics solutions using AWS services such as Amazon Web Services S3, Glue, Lambda, and Redshift, enabling data-driven decision-making across marketing and commercial functions
- Designed and implemented end-to-end ETL/ELT pipelines—from data ingestion to transformation and loading—integrating multiple data sources including CRM systems like Salesforce
- Built and optimized distributed data processing workflows using Python, PySpark, and SQL, ensuring scalability, reliability, and high-performance data processing
- Optimized and monitored data pipelines for performance, cost-efficiency, and reliability, leveraging cloud-native services and best practices
- Developed and maintained data models and warehouse structures to support scalable analytics and reporting use cases
- Collaborated with cross-functional teams including data scientists, analysts, and architects to translate business requirements into robust technical solutions
- Implemented data quality checks, governance practices, and monitoring frameworks to ensure data accuracy, consistency, and compliance
- Mentored junior engineers and drove best practices in data engineering, code quality, and pipeline design
- Communicated complex technical solutions and insights effectively to both technical and non-technical stakeholders
Data Engineer Intern
Amazon Robotics
05.2022 - 08.2022
- Designed and developed a data lineage solution using Amazon Web Services services including S3, Glue, Lambda, and Athena for the ORBIT (Optimized Robotics Business Insights and Tools) platform, enabling data visibility and traceability for BI engineers and data scientists
- Built a CLI-based tool to automate lineage tracking and metadata extraction, improving developer productivity and reducing manual effort
- Leveraged AWS Glue for metadata cataloging and ETL processing, Lambda for serverless orchestration, and Athena for querying lineage data efficiently
- Enabled downstream reporting and visualization through integration with Amazon QuickSight, supporting data-driven insights for robotics and analytics teams
- Delivered a scalable and cost-effective solution that was widely adopted across ORBIT and its users
Data Engineering Specialist
Johnson & Johnson
06.2018 - 06.2021
- Designed and implemented scalable data pipelines for CDC (Change Data Capture) across heterogeneous ERP systems (SAP, JD Edwards), enabling near real-time data ingestion into enterprise data platforms
- Built and optimized distributed data processing workflows using Databricks (PySpark, SQL) to handle high-volume data ingestion (~45TB/month) with efficient incremental load and CDC strategies
- Architected and supported Lakehouse data models (bronze, silver, gold layers) to enable downstream analytics and reporting with improved query performance and data usability
- Led end-to-end pipeline orchestration, monitoring, and alerting mechanisms, ensuring high availability and reliability of production workloads
- Implemented data quality checks, validation frameworks, and monitoring solutions, reducing data gaps and improving data reliability by ~85%
- Developed automation frameworks using Python and workflow orchestration tools (Airflow/Unix-based schedulers) to streamline pipeline development and reduce manual effort by 90%
- Tuned and optimized data pipelines for performance, scalability, and cost efficiency within distributed processing environments
- Led and mentored a team of 5–10 data engineers, driving best practices in code quality, reusable frameworks, and scalable architecture design
- Collaborated with cross-functional stakeholders to gather requirements, translate business needs into technical solutions, and ensure timely delivery of data products
Technical Lead DWH & Analytics
ResMed
07.2016 - 05.2018
- Led the Healthcare Informatics team at ResMed (Global leader in sleep apnea device manufacturing and respiratory devices)
- Analyzed advanced analytic objectives and built full stack healthcare analytics product enabling HMEs to view patient analytics and compliance rates for their populations
- The application allowed the customers to view patients that were at risk of not meeting compliance with 95% accuracy
- Designed and implemented metadata-driven object-oriented ETL engine to capture patients' sleep and respiratory improvements, enhancing clinical understanding of health outcomes
- It helped the Clinical Staff Team to understand patient's health improvements generating a revenue of more than 2 Mil
- Developed HI dashboard in Tableau to visualize key statistics related to patient data, supporting clinical decision-making
Senior ETL Developer
Larsen & Toubro Infotech
02.2015 - 06.2016
- Designed and implemented daily data load and reconciliation processes using shell scripts and stored procedures, ensuring data accuracy and availability.
- Monitored and optimized Informatica job run time, enhancing session performance.
- Demonstrated strong SQL skills in building custom datasets for analysis and reporting.
- Developed applications using NoSQL databases such as HBase, expanding data management capabilities.
Software Engineer
Wipro Technologies
07.2010 - 01.2015
- Proven expertise in building self-service data platforms for BI and Data visualization
- Enhanced Informatica jobs and SQL queries of data load/ingestion process to triple the load handling capacity
- Monitored long running SQL queries and made changes to reduce their run times by about 50%
- Developed ETL jobs to extract data from diverse sources and transform it for loading using Informatica PowerCenter.
- Handled SCD Type-1 and Type-2 dimensions using Informatica
- Streamlined processes to automate handling of ad-hoc file requests through shell scripting.
Education
Master of Science - Computer Science
New York University, Tandon School of Engineering
New York, NYMaster of Science - Software Engineering
BITS PILANI, Birla Institute of Technology And Science
IndiaBachelor of Science - Information Technology
SIES College, Mumbai University
IndiaSkills
- Programming Languages – Python, Java, Pyspark
- Databases - Oracle,Teradata,MS SQL Server
- Big Data Ecosystem - Hive ,Impala ,Confluent Kafka Delta Lake,Hadoop
- Cloud Stack - AWS -Lambda , S3 ,Glue, Athena,Dynamo DB, EMR,Redshift,Step functions, Cloud Formation
- Cloud Stack - Azure data factory, Azure data Lake Store, Azure synapse, Databricks ,Airflow
- ETL and BI skills - Informatica PowerCenter 8x, 9x ,10x ,Informatica Power exchange 96,101,102 and 104 , Informatica MDM , Snowflake , Power BI , Cognos, SAP BO
Accomplishments
- Data Lineage Solution
- Python Automation Efficiency
- Efficiency Improvement
- Healthcare Analytics Success
- Key Impact
Architected and implemented data lineage solution used by over 500 Orbit users monthly.
Reduced code development time by 90% through Python automation.
Optimized ETL workflows, improving job efficiency by 25% and lowering compute costs
Delivered a healthcare analytics platform generating $2M+ in revenue
Drove scalable, Cloud-based data ingestion and visualization solutions that improved reliability, accelerated decision-making, and enabled business stakeholders to act faster
Timeline
Lead Data Engineer
Analytic Partners
01.2023 - Current
Data Engineer Intern
Amazon Robotics
05.2022 - 08.2022
Data Engineering Specialist
Johnson & Johnson
06.2018 - 06.2021
Technical Lead DWH & Analytics
ResMed
07.2016 - 05.2018
Senior ETL Developer
Larsen & Toubro Infotech
02.2015 - 06.2016
Software Engineer
Wipro Technologies
07.2010 - 01.2015
Master of Science - Computer Science
New York University, Tandon School of Engineering
Master of Science - Software Engineering
BITS PILANI, Birla Institute of Technology And Science
Bachelor of Science - Information Technology
SIES College, Mumbai University