Summary
Overview
Work History
Education
Skills
Certification
Phone
Languages
Timeline
Generic

Pawan Sharma

Jersey City,NJ

Summary

Experienced Data Engineer with a strong focus on responsiveness and expertise in various aspects of the field. Skilled in managing ETL processes, monitoring database performance, troubleshooting issues, and optimizing data environments. Proficient in utilizing cloud services and database technologies to enhance data pipelines and facilitate data-driven decision making. Comfortable working independently and collaboratively, with exceptional communication skills to effectively convey complex concepts.

Overview

19
19
years of professional experience
1
1
Certification

Work History

Sr. Cloud Data Engineer / DWH Manager

EXL Services
02.2020 - Current
  • Company Overview: Domain: Healthcare (multiple clients)
  • Led the migration of on-premise healthcare data infrastructure to GCP, improving data processing speed by 40% and reducing operational costs by 35%
  • Built ETL solution using Azure cloud services, migrated on premise data to azure cloud using ADF component data flow, triggers, and data-bricks (PySpark)
  • Optimized Big-Query schemas and queries, resulting in a 50% improvement in query performance and cost reduction
  • Good understanding of optimization techniques
  • Implemented data governance and security best practices, ensuring compliance with GDPR and HIPPA regulations
  • Design, develop, and maintain ETL pipelines for healthcare data extraction from systems like Epic, CMS etc
  • Using Azure Data Factory, Azure Databricks, PySpark and Spark SQL, ensuring accurate and timely data loading
  • Monitored data pipeline performance and resolved data quality issues, maintaining 99.9% data accuracy
  • Develop, maintain, and optimize CI/CD pipelines using Jenkins and GitHub
  • Managed data extraction, transformation, and loading using AWS services (S3, Glue, Data Catalog, Redshift cluster, RDS)
  • Helped in developing python based ETL workflow on AWS which reduced the existing ETL tool cost $70K approximately
  • Migrated over 500 TB on prem healthcare data from legacy (Netezza, SAS & Hadoop) using GCP Big-query, Storage, Data-proc cluster & Airflow composer
  • Conducted thorough assessment and planning for migrating legacy data systems to cloud-based solutions
  • Executed data migration from on-premise databases to cloud platforms (GCP and AWS), ensuring data integrity and minimal downtime
  • Utilized tools like Sqoop and custom Python scripts for efficient data transfer
  • Coordinated with cross-functional teams to identify and resolve migration-related issues promptly
  • Documented migration processes and provided training to team members on new systems
  • Domain: Healthcare (multiple clients)

Manager Analytics

Paytm
04.2013 - 01.2020
  • Company Overview: Domain: Digital Services & Payment
  • Transitioned from MySQL for OLTP and OLAP to Big Data using Hadoop and Hive ecosystem
  • Managed data ingestion from various sources including RDBMS, AWS S3, and real-time data streaming from mobile apps/websites
  • Automated repetitive tasks using scripting languages and workflow automation tools, reducing manual processes
  • Mentored junior team members in technical skills and professional development
  • Led the migration of transactional data from MySQL to Hadoop ecosystem, enhancing data processing capabilities
  • Ensured data consistency and quality during the migration process through rigorous testing and validation
  • Developed and executed data migration strategies to ensure smooth transitions and minimal disruptions to business operations
  • Collaborated with stakeholders to identify requirements and address potential risks associated with data migration projects
  • Domain: Digital Services & Payment

Senior Data Analyst

One97 Communication
07.2010 - 03.2013
  • Streamlined data analysis workflows, increasing efficiency, and accelerating decision-making processes
  • Enhanced data accuracy through stringent data validation processes and quality control measures
  • Utilized business objects, business intelligence, and other reporting tools to extract data from data solutions and data warehouses
  • Helped telecom vendors to achieve valued customers for VAS and USSD services

Customer Care Executive

EXL Services
02.2007 - 12.2009
  • Managed customer database in SAP, proactively identifying and resolving issues to reduce customer complaints
  • Assisted in training new team members to ensure high levels of customer care expertise

Data Analyst

Competent Software
09.2005 - 01.2007
  • Analyzed US based mortgage documents like deeds, insurance for property sale/purchase/taxes

Education

Bachelor of Science - Mathematics, Physics, Chemistry

HNB Garhwal University
Srinagar, Garhwal
06.2003

Skills

  • GCP
  • AWS
  • AZURE
  • Big-Query
  • Redshift
  • Hadoop
  • Hive
  • Databricks
  • Python
  • PySpark
  • Pandas
  • SQL
  • Jenkins
  • GIT
  • Apache Airflow
  • Oozie
  • Control-M
  • Azure Data Factory
  • AWS Glue
  • Azure Data Lake
  • GCP Dataflow
  • Pub/sub
  • Sqoop
  • Talend Studio
  • Tableau
  • Quick-Sight
  • ETL development
  • Data warehousing
  • Data modeling
  • Data pipeline design
  • Data migration
  • Spark framework
  • SQL expertise
  • Data governance
  • Hadoop ecosystem
  • SQL and databases

Certification

Google Cloud Associate, 09/01/23

Phone

+1-(973)-391-7161, +91-9910400197 (WhatsApp)

Languages

English
Full Professional
Hindi
Full Professional

Timeline

Sr. Cloud Data Engineer / DWH Manager

EXL Services
02.2020 - Current

Manager Analytics

Paytm
04.2013 - 01.2020

Senior Data Analyst

One97 Communication
07.2010 - 03.2013

Customer Care Executive

EXL Services
02.2007 - 12.2009

Data Analyst

Competent Software
09.2005 - 01.2007

Bachelor of Science - Mathematics, Physics, Chemistry

HNB Garhwal University

Similar Profiles

CREATE PROFILE
Pawan Sharma