Summary
Overview
Work History
Education
Skills
Accomplishments
Timeline
Generic

Durga Priya Vanam

Herndon,VA

Summary

Accomplished Big Data Engineer with expertise in Apache Spark and data migration at Freddie Mac. Successfully led the transition to Stratus, enhancing data integrity and reducing downtime. Proven ability to collaborate with cross-functional teams, delivering innovative data solutions that drive business intelligence and reporting efficiency.

Overview

7
7
years of professional experience

Work History

Big Data Engineer

Freddie Mac
McLean, VA
02.2023 - Current
  • Led the end-to-end data migration process from Hadoop to Stratus, ensuring data integrity and minimal downtime during transition.
  • Developed and optimized big data applications using Hadoop, Spark, and Sqoop to support scalable data processing workflows.
  • Designed and implemented robust data pipelines to ingest data from diverse sources into Hadoop ecosystems using Sqoop and Spark.
  • Built and maintained custom data processing solutions within Linux environments to enhance processing efficiency and system stability.
  • Architected and deployed data storage solutions using HDFS and Hive, aligning with distributed data management best practices.
  • Authored and optimized Hive queries for analytical reporting, employing bucketing and partitioning techniques to boost query performance.
  • Designed and maintained ETL processes utilizing Python, PySpark, and Scala to transform and load large-scale datasets.
  • Created dynamic data visualizations and dashboards using Tableau and Power BI to support business intelligence and reporting needs.
  • Developed and maintained unit tests, regression tests, and integration tests to ensure data quality and pipeline reliability.
  • Collaborated cross-functionally with data analysts, data scientists, and business stakeholders to deliver tailored data solutions.
  • Documented data workflows, processing frameworks, and system architecture for operational transparency and knowledge sharing.
  • Provided production support for big data systems, performing root cause analysis and troubleshooting to resolve data issues efficiently.
  • Actively monitored industry trends and emerging technologies to drive innovation in big data practices and toolsets.
  • Participated in peer code reviews, offering constructive feedback to uphold coding standards and promote team learning.

Data Engineer

Accenture
Hyderabad, India
12.2018 - 08.2021
  • Led a key role in the design and implementation of a client Enterprise Data Warehouse (EDW) and Operational Data Store (ODS), orchestrating the seamless migration of data from NAS and outdated legacy CDW data marts, resulting in a 25% reduction in query response times and enhancing overall data availability for reporting.
  • Developed and executed intricate HIVE queries tailored to meet the specific data retrieval and analysis needs of a prominent healthcare client.
  • Developed shell scripts and SQL stored procedures to streamline and optimize data processing within ETL workflows, significantly reducing job complexity and enhancing overall efficiency in data transformation tasks.
  • Automated build and deployments using CI/CD tools - Bitbucket, Git.

Education

Master of Science - Computer Science - GPA: 3.9

University of Missouri
Kansas City, MO
12.2022

Bachelor of Science - Electrical Engineering - GPA: 3.7

Kakatiya University College of Engineering & Technology
Warangal, India
01.2018

Skills

  • Programming: Python, Java
  • Big Data Technologies: Apache Spark, Spark-SQL, Hadoop, Hive
  • ERP: SAP PI, SAP R3
  • Database: SQL & NoSQL (SQL Server, MySQL, DB2, Oracle)
  • Job Scheduling: Control-M, Autosys
  • Reporting Tools: Tableau, Power BI
  • ETL Tools: Talend, Attunity, Qlik, IICS
  • Agile: Jira, IBM RTC
  • Cloud Services: AWS (S3, EC2, DynamoDB, Glue, CloudWatch, Step Functions, Lambda, DataSync, API Gateway)
  • Logging and Monitoring: Kibana, Elasticsearch, Splunk
  • Data Warehouses: Snowflake, Dremio

Accomplishments

  • Conducted a robotic workshop on sensor-based maze-solving cars at IIT-Delhi (ROBOTRYST'15).
  • Internship at Indian Railways emphasizing the 'Power and Control Circuit of WAG-7 locomotive.'

Timeline

Big Data Engineer

Freddie Mac
02.2023 - Current

Data Engineer

Accenture
12.2018 - 08.2021

Master of Science - Computer Science - GPA: 3.9

University of Missouri

Bachelor of Science - Electrical Engineering - GPA: 3.7

Kakatiya University College of Engineering & Technology

Similar Profiles

CREATE PROFILE
Durga Priya Vanam