Summary
Overview
Work History
Education
Skills
Websites
Timeline
Generic

Venkata Mangipudi

Sr. Data Engineer
Leander,TX

Summary

  • Over 8 years of experience in IT with strong expertise in project and account management, and full lifecycle software development across diverse enterprise systems.
  • Proficient in Big Data technologies with hands-on experience developing, implementing, and maintaining applications using Hadoop, Azure, Databricks (PySpark), Python, T-SQL, PL/SQL, Azure Data Factory (ADF), Azure Synapse, Azure Data Lake Gen2, Azure DevOps, and SSIS.
  • Skilled in designing and visualizing data models (both dimensional and relational), including Star and Snowflake schemas, ensuring consistency and clarity in database design.
  • Extensive experience in data profiling, mapping, integration, transformation, cleansing, validation, and loading for large-scale analytics systems.
  • Built scalable ETL pipelines integrating diverse data sources into unified views using ADF, Databricks, and Azure Synapse.
  • Ensured data quality and integrity by identifying and resolving inconsistencies, duplicates, and anomalies through structured validation checks.
  • Optimized database performance via indexing, partitioning, and performance tuning, improving query efficiency and resolving processing bottlenecks.
  • Translated complex business requirements into effective data models and storage strategies through collaboration with cross-functional teams.
  • Hands-on experience with Delta Live Tables (DLT) in Databricks, including ETL job configuration and monitoring.
  • Strong analytical skills in data exploration and reporting using Databricks, Spark, Python, T-SQL, PL/SQL, and Azure DW.
  • Experienced in building dashboards and KPIs using Power BI and SSRS for business performance tracking.
  • Skilled in real-time data analytics using Spark Streaming and data transformation using Hive.
  • Working knowledge of AWS services including S3, EMR, EC2, and Lambda for scalable cloud-based data processing.
  • Adept in Python scripting for data extraction and transformation using built-in data structures (Lists, Tuples, Dictionaries, Sets).
  • Experience managing large, complex databases across MySQL, SQL Server, Oracle, and Hive.
  • Proficient in version control tools like Git, Azure DevOps, and TFS for collaborative development and deployment.
  • Knowledgeable in Agile and SDLC methodologies for iterative development and project delivery.
  • Capable of managing ETL architecture, database optimization, and full application lifecycle support.
  • Strong ability to understand complex systems, solve analytical problems, and build reusable, scalable data solutions.
  • Experienced in acquiring data from multiple sources and maintaining robust data infrastructure.
  • Background in healthcare analytics, including trend analysis and reporting across claims, membership, and care/utilization data.

Overview

10
10
years of professional experience
4
4
years of post-secondary education

Work History

Sr. Data Engineer

Apple
Austin, Texas
02.2023 - Current
  • Involved in complete software development life cycle (SDLC) like Requirement Analysis and Specification, technical design, and Implementation.
  • Implemented advanced PySpark scripts in Azure Databricks to perform data transformations, cleansing, and aggregation, improving ETL efficiency by 30%.
  • Leveraged Delta Lake on Azure Databricks for efficient data storage, ensuring ACID compliance and enabling real-time analytics.
  • Led and assisted in developing Supply planning applications using SQL for data extraction, transformation, and aggregation for analyzing & transforming data to load, which helps users plan efficiently.
  • Utilized Azure Data Factory to orchestrate Databricks pipelines and integrate with other cloud services for end-to-end ETL workflows.
  • Worked extensively with Azure Data Explorer to analyze large datasets and extract meaningful insights.
  • Designed and implemented data transformation workflows in Azure Databricks, optimizing processing times and ensuring efficient handling of large datasets.
  • Implemented robust error handling, logging, and monitoring mechanisms in Databricks to ensure pipeline reliability and fault tolerance.
  • Formulating DAX Expressions to create Date to yearly columns and implemented partitions in Tabular Models for Power BI.
  • Creating stored procedures and SQL queries to pull data into the power pivot model.
  • Using Power BI Desktop to create KPI scorecards, dashboards, and visual reports for business users that boost productivity by 15%.
  • Using SQL writing Joins and sub-queries for complex queries involving multiple tables from different databases.
  • Created database and database objects like tables, stored procedures, views, triggers, rules, defaults user-defined data types, and functions with T-SQL to facilitate data consistency.
  • Developed and maintained CI/CD pipelines using Azure DevOps and Jenkins, automating SQL-based application build and deployment processes.
  • Identifying/Isolate the performance bottlenecks, and providing recommendations to improve performance.
  • Provided support for database systems, including PostgreSQL, MS SQL, Oracle and MySQL.
  • Extensive knowledge on Databricks Unity Catalog to manage data governance, access control, and security policies across multiple Azure data platforms.

Sr. Data Engineer

Kohl's
Austin, Nevada
08.2021 - 01.2023
  • Designed the AZURE Data Lakes and AZURE Data Warehouse's future state architecture, which reduced processing time by 15% and significantly shortened the time needed to refresh data for reports.
  • Migrated legacy ETL processes to Azure Databricks, reducing processing times by 40% and improving scalability.
  • Integrated Azure Databricks with other Azure services, including Blob Storage, Data Lake, and Azure Synapse Analytics, to enable seamless data processing and analytics.
  • Developed and maintained scalable data pipelines using Python and Apache Spark, enabling real-time data processing and analytics.
  • Contributed to AZURE End-to-end architecture and created meaningful dashboards with MS Power BI.
  • This led to a smooth transfer of legacy data from an on-premises data warehouse to a cloud-based solution and enhanced data accessibility.
  • Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.

Data Engineer

Payless ShoeSource
Sacramento, CA
09.2019 - 07.2021
  • Designed, defined, and planned a database according to the documentation and business needs.
  • Developed various views and stored procedures to update the data using another database.
  • Created heavy-duty TSQL to join data from various views and functions, including left join, inner join, and outer join.
  • Worked on different types of transformations that are available in Power BI query editor.
  • Created packages using SSIS for data extraction from Flat Files, Excel Files, OLEDB to SQL Server.
  • Wrote Python routines to log into websites and fetch data for selected options.
  • Collaborated with business stake to integrate various data sources into Business Objects, ensuring accurate and timely reporting by leveraging ETL processes.
  • Created Datasets in T-SQL, and stored procedures for Reporting services.

Sr. SQL Developer

Alluma
Sacramento, CA
04.2017 - 08.2019
  • Developed an Operational Dashboard on SSRS to provide performance Key Performance Indicators (KPIs) of different govt agencies within the Arizona Govt. for weekly review meetings and helped improve operational efficiency by 15%.
  • Developed various T-SQL stored procedures, triggers, views, and adding/changing tables for data load, transformation, and extraction.
  • Developed Python ETL services for data loading, file parsing, and capturing audit data.
  • Created Packages by testing and Cleaning the Standardized Data by using tools in Data Flow Transformations (Data Conversion, Export Column, Merge join, Sort, Union All, Conditional Split, and more) for existing/new packages.
  • Writing different validation Scripts to check the accuracy of data.
  • Created and scheduled SSIS packages for running AM and PM feeds from various departments and multiple servers and resources to Development Servers. Logged various packages as well as individual tasks using SQL server log providers like text files, SQL server database, trace files, XML file.

SQL Developer

WIPRO
Mysore, India
10.2015 - 02.2017
  • Analyzed the current business processes and recommended/developed ETL solutions to meet the client's needs.
  • Developed various T-SQL stored procedures, triggers, views, and adding/changing tables for data load, transformation, and extraction.
  • Validated Data Integrity between SQL Database and Oracle PL/SQL Database by conducting Unit testing, Integration testing, User Acceptance testing.
  • Involved in optimizing code and improving efficiency in databases including re-indexing, updating statistics, recompiling stored procedures, and performing other maintenance tasks. Involved in performance tuning of the slow-running queries and stored procedures.

Education

Bachelor of Science - Electrical Engineering

GITAM University
Visakhapatnam
06.2011 - 05.2015

Skills

  • Amazon Redshift

  • Google BigQuery

  • Microsoft Azure

  • Amazon S3

  • Teradata

  • IBM Db2

  • Apache Airflow

  • Informatica

  • Talend

  • Workato

  • Microsoft SSIS

  • EMR

  • MuleSoft

  • Apache Spark

  • Kafka

  • Amazon RDS

  • MySQL

  • Oracle Database

  • PostgreSQL

  • Microsoft SQL Server

  • ERwin Data Modeler

  • Oracle SQL Developer Data Modeler

  • Lucidchart

  • ER/Studio

  • Tableau

  • Microsoft Power BI

  • MicrosStrategy

  • SAP Lumira

  • Snowsight

  • Erwin Data Intelligence

  • SAP Master Data Governance

  • Git

  • AWS CodeCommit

  • Azure DevOps

  • Visual Studio Code

  • Confluence

  • Jira

  • OneDrive

  • Gantt Charts

  • Scrum

  • Kanban

  • Microsoft Project

  • Asana

Timeline

Sr. Data Engineer

Apple
02.2023 - Current

Sr. Data Engineer

Kohl's
08.2021 - 01.2023

Data Engineer

Payless ShoeSource
09.2019 - 07.2021

Sr. SQL Developer

Alluma
04.2017 - 08.2019

SQL Developer

WIPRO
10.2015 - 02.2017

Bachelor of Science - Electrical Engineering

GITAM University
06.2011 - 05.2015

Similar Profiles

CREATE PROFILE
Venkata MangipudiSr. Data Engineer