Nikhila Pedapalli
Boston,USA
Summary
Seasoned Senior Data Engineer with 7+ years background in developing, testing, and maintaining data architectures. Possess strong skills in database management systems, Big Data processing frameworks, data modeling and warehousing. Have successfully led teams in creating innovative data solutions to improve system efficiency and business decision-making processes. Demonstrated impact through enhanced data availability and accuracy in previous roles.
Overview
8
8
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
American Express
Boston, MA
06.2024 - Current
- Architected and deployed a real-time transaction processing system using Azure Event Hubs and Kafka, achieving 99.99% uptime and processing over 2 million transactions per hour
- Delivered a cloud-native ETL solution on Azure Data Factory that successfully processed 10TB+ of financial data daily, reducing processing time by 40%
- Executed a database optimization project using Azure Synapse Analytics, resulting in 35% faster query performance and $200K annual cost savings
- Implemented an Azure ML-based MLOps framework for model deployment that increased model release velocity by 60% and improved model accuracy by 25%
- Established a comprehensive data governance system using Azure Purview that ensured compliance with financial regulations while maintaining data accessibility for analytics teams
Data Engineering Lead
Zipply Fiber
Remote
04.2023 - 05.2024
- Architected and launched an Azure Data Lake Storage Gen2 solution that consolidated 15+ data sources, enabling cross-functional analytics and reducing data silos by 75%
- Built a real-time network performance monitoring system using Azure Functions and Event Hubs that identified service degradation 15 minutes faster than previous systems
- Delivered an automated data quality framework using Azure Data Factory that reduced data anomalies by 40% and prevented 28 potential data incidents
- Completed migration of 8TB legacy data warehouse to Azure Synapse Analytics, resulting in 65% improved query performance and 30% reduction in operational costs
- Established CI/CD pipelines using Azure DevOps and ARM templates, decreasing deployment time from days to hours and reducing configuration errors by 90%
Senior Data Engineer (Contract)
Mayo Clinic
Remote
02.2023 - 04.2023
- Architected and implemented HIPAA-compliant data pipelines on Azure Data Factory that securely processed 50TB+ of patient data, enabling advanced analytics while maintaining 100% regulatory compliance
- Developed a real-time patient monitoring system using Azure IoT Hub and Event Hubs that reduced critical alert response time by 40% and improved patient outcomes by 15%
- Engineered a machine learning pipeline using Azure Machine Learning that accurately predicted patient readmission risks with 85% accuracy, helping reduce readmission rates by 23%
- Created an automated data quality monitoring system with Azure Data Quality Service that identified and flagged data anomalies in clinical records with 99% accuracy, improving data reliability for research teams
- Established a comprehensive data governance framework with Azure Purview that streamlined access to clinical data for research while maintaining strict security and privacy controls
Data Engineer
BELK Insurance
Remote
07.2022 - 01.2023
- Developed and implemented Azure Data Factory and Blob Storage-based data pipelines that integrated claims, policy, and customer data, resulting in a unified data platform that improved reporting accuracy by 45%
- Established comprehensive data governance protocols for PII handling using Azure Information Protection that ensured 100% compliance with HIPAA and industry regulations
- Created interactive Power BI dashboards connected to Azure data sources that reduced claims processing time by 30% through improved visibility into process bottlenecks
- Delivered real-time reporting systems using Azure Analysis Services that decreased executive decision-making time by 25% and improved strategic planning
- Optimized ETL processes that reduced processing time by 45% and decreased Azure infrastructure costs by $120K annually
Data Engineer (Contract)
State Farm Insurance
Remote
02.2021 - 06.2022
- Architected a comprehensive Azure-based data pipeline that processed 5TB claims daily, reducing processing time by 35%
- Implemented Azure Functions for real-time fraud detection that identified suspicious patterns and saved $3.2M in potential fraudulent claims annually
- Developed a predictive maintenance system for claims processing infrastructure using Azure Machine Learning that reduced system downtime by 45%
- Created automated disaster recovery solution using Azure Site Recovery, reduced recovery time objective from days to hours
- Led integration of 8 disparate data sources into a unified Azure Data Lake that improved cross-functional analytics capabilities by 60%
Data Engineer
URBN (Urban Outfitters)
Remote
07.2019 - 12.2020
- Designed and implemented GCP BigQuery data warehouse solution that consolidated retail analytics from 5 different systems, enabling unified reporting and improving decision-making speed by 60%
- Developed ETL pipelines using Cloud Dataflow that successfully processed and integrated 3TB of e-commerce, inventory, and customer data daily with 99.9% reliability
- Created data models in BigQuery for product performance analysis that identified $2.5M in inventory optimization opportunities and improved seasonal planning
- Built automated reporting system using Google Data Studio that reduced inventory stockouts by 15% and improved forecast accuracy by 20%
- Implemented Cloud Monitoring-based data quality monitoring solution that caught 98% of data anomalies before they impacted business operations
Data Analyst
AppIcon IT
Bengaluru, India
06.2017 - 07.2019
- Implemented GCP-based data collection processes across 8 different source systems that enabled comprehensive business intelligence reporting and improved data accessibility by 65%
- Developed ETL processes using Cloud Dataprep that successfully transformed raw data into actionable insights, reducing report generation time from days to hours
- Created interactive Data Studio dashboards for sales and marketing teams that increased sales team productivity by 25% and improved lead conversion rates by 18%
- Authored 200+ BigQuery SQL queries that successfully extracted and analyzed customer behavior patterns, identifying $1.5M in new revenue opportunities
- Deployed machine learning models using Google AI Platform that improved customer segmentation accuracy by 40% and enhanced targeted marketing campaigns
Education
Master of Science - Information Systems
Northeastern University
Boston, MA, USABTech - Computer Science and Engineering
Gitam University
Bengaluru, IndiaSkills
- Azure data services
- Data engineering and analytics
- Cloud computing technologies
- Database management systems
- Big data processing frameworks
- Data visualization tools
- Machine learning frameworks
- Container orchestration and management
Certification
- AWS Certified Data Analytics Specialty
- Google Cloud Professional Data Engineer
- Microsoft Azure for Data Engineering by Microsoft
Projects
GenBI - Agentic AI for Business Intelligence, Architected and developed a full-stack agentic AI system that revolutionized data analysis workflows by enabling natural language querying of complex datasets., Engineered a production-grade Streamlit application integrating OpenAI GPT-4 with custom-designed agents for query classification, data manipulation, and visualization generation., Implemented sophisticated data processing pipeline handling 20+ data formats with automated schema detection and transformation capabilities., Created an interactive chat interface with parallel processing features that reduced analysis time by 85% compared to traditional BI tools., Designed and optimized visualization algorithms that automatically selected the most appropriate chart types based on data characteristics and query intent., Deployed the solution on AWS using containerization and orchestration, achieving 99.5% uptime and supporting concurrent analysis of datasets up to 1GB., Received academic excellence award and was selected for the university's innovation showcase.
Timeline
Senior Data Engineer
American Express
06.2024 - Current
Data Engineering Lead
Zipply Fiber
04.2023 - 05.2024
Senior Data Engineer (Contract)
Mayo Clinic
02.2023 - 04.2023
Data Engineer
BELK Insurance
07.2022 - 01.2023
Data Engineer (Contract)
State Farm Insurance
02.2021 - 06.2022
Data Engineer
URBN (Urban Outfitters)
07.2019 - 12.2020
Data Analyst
AppIcon IT
06.2017 - 07.2019
Master of Science - Information Systems
Northeastern University
BTech - Computer Science and Engineering
Gitam University
Similar Profiles
Nemecio ContrerasNemecio Contreras
(Fraud Prevention) at American Express(Fraud Prevention) at American Express
Destinez CharlesDestinez Charles
Credit and Collections Specialist at American ExpressCredit and Collections Specialist at American Express
Dewanka SardanaDewanka Sardana
Sr BST Analyst at American ExpressSr BST Analyst at American Express
SHAQUILE MAJAWAHSHAQUILE MAJAWAH
Data Center Infrastructure Technician at American ExpressData Center Infrastructure Technician at American Express
Emily DunbarEmily Dunbar
Server at City WineryServer at City Winery