Summary
Overview
Work History
Education
Skills
Accomplishments
Certification
Timeline
SoftwareDeveloper

Varshith Dandu

Summary

Motivated data engineer around 3 years of IT experience. Proficient in data manipulation and transformation techniques, with a strong understanding of SQL and ETL processes. Solid foundation in data warehousing concepts and cloud platforms like AWS or Azure. Hardworking and passionate job seeker with strong organizational skills and quick learner with a passion for solving complex problems and a strong attention to detail. Ready to help team achieve company goals.

Overview

4
4
years of professional experience
1
1
Certification

Work History

Data Engineer

Express Scripts
01.2022 - Current
  • Extracted data from external sources such as Teradata, SQL Server, Mongo DB and API's
  • Built Data pipelines using Azure Data Factory by connecting to different database sources using JDBC connectors.
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Created automated Databricks workflow to run multiple data loads or increasing parallel processing and performed Transformations
  • Ingested Curated data from Delta Lake to Azure Data Lake Gen2 which was built on top of Azure Blob Storage
  • Stored the Curated structured data in Azure synapse for analytics for visualization.
  • Created visualization dashboards using PowerBI based on the business use cases.
  • Developed Spark jobs using PySpark and Spark SQL to perform data cleaning, filtering, aggregation, and enrichment on batch data.
  • Monitored and maintained the performance and health of the data pipelines and Spark jobs by using Azure Monitor.
  • Used Azure Key Vault 2.0 to store and manage passwords, API keys, and ensuring strong security measures.
  • Leveraged Azure DevOps to implement CI/CD pipelines utilizing Azure Repository as the version control system by managing branches and implementing automated build, test, and deployment.

Data Analytics Engineer

Fruitstone
Hyderabad, Telangana
01.2019 - 12.2019
  • Created dashboards for visualization using Tableau for Business Analysts for daily, weekly, bi-weekly, monthly and quarterly basis
  • Migrated data from on premises to AWS Redshift performing an end to end ETL Operations
  • Developed pipelines using AWS Glue to extract data from various external sources and load it into S3
  • Integrated the data and transfer from S3 buckets to AWS Redshift data warehouse
  • Used AWS Athena for querying the data stored in S3 Buckets
  • Written Pyspark scripts to remove null values, duplicate data and Spark SQL for querying
  • Tracked and analyzed the performance of data pipelines using AWS Cloud Watch on day to day basis
  • Used AWS Simple Notification Service (SNS) for any pipeline failures and AWS IAM for access control of data pipelines.

Education

Masters - Computer science

Missouri State University
12.2021

Bachelors - Computer Science

Mahatma Gandhi Institute of Technology
04.2019

Skills

  • Azure Cloud Services : Azure Data Factory, Azure Synapse Analytics, Data Lake, Blob Storage, , Azure Databricks, Azure Data Analytics,
  • AWS Cloud Services : EMR, Redshift, S3, Athena, Glue, Cloud Watch, SNS, IAM
  • ETL/BI : PowerBI, Tableau, SSRS, SSAS, HBase, Dynamo DB, Mongo DB
  • Hadoop/Big Data Technologies : Hadoop, MapReduce, HDFS, Hive, Sqoop, Spark, Nifi,
  • Programming & Scripting : Python, Spark, SQL, Shell Scripting,
  • Databases : Oracle, MY SQL, Teradata, PostgreSQL, SQL Server
  • Version Control : GitHub, Azure DevOps

Accomplishments

  • Data Science Certification: Missouri state University
  • Microsoft Certified: Azure Data Engineer DP-203: https://www.credly.com/badges/91286092-ae91-425f-9551-dd759acdd9b7?source=linked_in_profile

Certification

Microsoft Certified: Azure Data Engineer DP-203:

https://www.credly.com/badges/91286092-ae91-425f-9551-dd759acdd9b7?source=linked_in_profile

Timeline

Data Engineer

Express Scripts
01.2022 - Current

Data Analytics Engineer

Fruitstone
01.2019 - 12.2019

Masters - Computer science

Missouri State University

Bachelors - Computer Science

Mahatma Gandhi Institute of Technology

Similar Profiles

CREATE PROFILE
Varshith Dandu