Summary
Overview
Work History
Education
Skills
Websites
Certification
Personal Information
Work Availability
Accomplishments
Interests
Languages
Timeline
AdministrativeAssistant
Avinash Khanderi

Avinash Khanderi

Bentonville,Arkansas

Summary

Transforming complex data challenges into actionable insights with 6+ years of experience in scalable architecture design and cloudbased solutions. Adept at leveraging innovative strategies to optimize data workflows, streamline analytics, and drive business value. Recognized for delivering impactful solutions that empower decision-making and fuel organizational growth. Ready to leverage experience and expertise to drive innovation and success in your organization.

Overview

7
7
years of professional experience
1
1
Certification

Work History

Senior Data Engineer

Walmart
Bentonville, AR
09.2024 - Current
  • Led and mentored a team of data engineers, driving innovative POC initiatives and fostering team growth.
  • Oversaw production deployments, managed change requests, and facilitated seamless integration by collaborating with crossfunctional teams and stakeholders.
  • Designed and optimized ETL pipelines to migrate data from RDBMS and HDFS to GCS buckets using Scala, Python, and Spark, improving efficiency by 40%.
  • Managed and secured data storage in GCS buckets, implementing efficient access controls and retrieval mechanisms.
  • Configured and automated CI/CD pipelines with Maven, streamlining processes & reducing manual interventions.
  • Automated complex workflows using Apache Airflow and Automic, optimizing pipeline execution and reliability.
  • Enhanced data storage and governance by implementing Delta tables, supporting versioning and ensuring ACID compliance.
  • Optimized Dataproc clusters and Spark jobs, reducing resource usage and improving processing times by 20%.
  • Developed notebooks integrated with Unity Catalog, enabling centralized governance and Delta Sharing.
  • Conducted cost analysis for cloud operations, implementing strategies to reduce expenses while maintaining performance.
  • Created BigQuery UDFs for advanced transformations & implemented authorized views, improving data governance by 30%.
  • Enhanced query performance by 25% through SQL and Spark SQL optimization techniques.
  • Reviewed and optimized code for adherence to best practices, mentoring team members & improving overall quality metrics.
  • Automated data validation in GCP buckets using Spark shell scripts, reducing manual effort by 50%.
  • Documented data pipelines and processes to improve team knowledge sharing and expedite onboarding.
  • Environment: Scala, Python, GCS, GCP, Delta Lake, BigQuery, Airflow, Automic, Databricks, Jenkins, Maven, Unity Catalog

Senior Data Engineer

VISA
Austin, TX
06.2022 - 09.2024
  • Designed and deployed Databricks Notebooks using PySpark, Scala, and Spark SQL for complex data transformations.
  • Managed diverse data formats (CSV, JSON, XML, Parquet) on HDFS, ensuring efficient storage and processing.
  • Orchestrated highly efficient Azure Data Factory (ADF) pipeline reducing data ingestion & transformation time.
  • Leveraged Delta Sharing for controlled access to datasets, ensuring compliance with privacy regulations.
  • Utilized Databricks for managing data pipelines and workflows in Unity catalog, enhancing governance and visibility.
  • Integrated Apache Airflow with ADF for efficient data orchestration and workflow management.
  • Led migration of VISA's on-premises data warehouse to Data Lake using Databricks, reducing costs by 35%.
  • Developed and optimized PowerBI data models, defining relationships, hierarchies, using DAX and schema methodologies.
  • Utilized Amazon S3 buckets for efficient storage and management of data assets within AWS ecosystem.
  • Optimized data integrity with Hive and PostgreSQL, reducing discrepancies for confident decision-making.
  • Streamlined ELT pipelines using ADF V2, achieving faster data ingestion into ADLS Gen2 and 30% cost reduction.
  • Developed and deployed SSAS cubes/Azure Analysis Services in Azure, resulting in faster data processing and reduction in manual scheduling efforts.
  • Environment: MS SQL Server/SSMS, ADF, Power BI Desktop, JIRA, Azure SQL Database, SSIS, ADLS, S3, Python, GIT, Spark, Kafka, Hive, Blob Storage, Event hub, Databricks, Scala, Azure DevOps, Wiki, Splunk.

Data Engineer

TechPlus Solutions
United Kingdom
01.2020 - 04.2022
  • Migrated data from legacy sources to HDFS using Python and PySpark.
  • Developed and maintained PySpark applications for large-scale dataset processing, improving efficiency by 30%.
  • Created interactive dashboards, resulting in improvement of report accessibility & saving users 10 hours per week.
  • Utilized Tableau for data visualization, enhancing efficiency by 20% and reducing decision-making time.
  • Leveraged Amazon S3 for secure and scalable storage of data assets, ensuring high availability and seamless integration with AWS services.
  • Utilized AWS Databricks for Spark-based ML workloads, reducing data processing time.
  • Utilized Apache Hive for SQL-like querying and analysis of large datasets stored in distributed systems.
  • Developed and maintained ETL processes using Informatica and SSIS.
  • Environment: Tableau, AWS, R Programming, S3, SQL, PySpark, Hive, SSIS, Informatica, Python.

Big Data Analyst

IBM
Hyderabad
07.2019 - 12.2019
  • Implemented Hadoop and Spark solutions, reducing data ingestion time and computational costs resulting in faster business insights.
  • Automated report generation using SQL, Python, R, DW concepts, & advanced Excel, saving 20 hours of manual work.
  • Streamlined data extraction and management with Data Analysis to improve performance and resource savings.
  • Optimized inventory levels using SQL and Python, reducing overall inventory costs in the supply chain.
  • Integrated AWS Sage Maker with real-time Kinesis streams for predictive analytics, improving decision-making speed.
  • Designed a cost-efficient AWS architecture and deployed CI/CD pipelines, resulting in savings of cloud expenses and reduction in deployment times.
  • Analyzed data trends and patterns from multiple sources, performing predictive statistics on warehouse data.
  • Conducted cluster analysis and decision tree design in R, integrated with Tableau to visualize Transportation data.
  • Environment: AWS, R Programming, SQL, MapReduce, Sqoop, Spark, Kafka, Pig, SQL Server 2012, Oozie.

Data Analyst

Amazon
India
08.2018 - 07.2019
  • Conducted in-depth data profiling and analysis using complex SQL across multiple source systems, focusing on network data optimization.
  • Designed and implemented advanced SQL queries and stored procedures to support inventory management and improve sales profitability.
  • Built and managed interactive dashboards in Tableau, integrating data from multiple sources to streamline visualization and reporting.
  • Developed and tested PL/SQL scripts for data validation and report generation, ensuring data quality and consistency.
  • Automated data integration workflows using SQL Server Integration Services (SSIS) and optimized data ingestion pipelines for heterogeneous sources, including Oracle.
  • Created and managed SAS datasets and data marts for reporting, developing tables, graphs, and dashboards to support decisionmaking processes.
  • Conducted business intelligence analysis using OLAP tools, transforming complex datasets into actionable insights for the network team.
  • Utilized Python to automate repetitive tasks and enhance data preparation workflows, improving operational efficiency.
  • Worked closely with cross-functional teams to streamline data workflows and ensure accurate, actionable analytics.
  • Environment: SQL Server, Tableau, Python, SAS, SSIS, MS Access, MS Excel, Visual Studio, Erwin.

Education

MS - Information Studies

Trine University
01.2023

MS - Engineering Management

Nottingham Trent University
01.2022

BTech - Mechanical Engineering

Vellore Institute of Technology
04.2018

Skills

  • Azure
  • AWS
  • Scala
  • GCP
  • Python
  • ADF
  • Databricks
  • Airflow
  • Glue
  • T-SQL/PL-SQL
  • Oracle
  • Teradata
  • Synapse
  • Redshift
  • Snowflake
  • R
  • Spark
  • Hadoop
  • Oozie
  • Pig
  • PowerBI
  • Dbt
  • Kafka
  • Tableau
  • Swagger
  • Kubernetes
  • IntelliJ
  • NLP
  • Scikit-learn
  • TensorFlow
  • Shell
  • SAFe
  • Azure DevOps
  • Jira
  • Terraform
  • Azure Purview
  • Collibra
  • Docker
  • Git

Certification

  • Databricks Certified Data Engineer Associate
  • Microsoft Certified - Azure Data Engineer Associate
  • Microsoft Power BI

Personal Information

Title: Senior Data Engineer

Work Availability

monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse

Accomplishments

Certifications

IEEE Senior Member

Interests

Books

Adventure

Travelling

AI

Languages

English
Native or Bilingual
Telugu
Native or Bilingual
Hindi
Full Professional
Tamil
Full Professional
French
Limited Working

Timeline

Senior Data Engineer

Walmart
09.2024 - Current

Senior Data Engineer

VISA
06.2022 - 09.2024

Data Engineer

TechPlus Solutions
01.2020 - 04.2022

Big Data Analyst

IBM
07.2019 - 12.2019

Data Analyst

Amazon
08.2018 - 07.2019

MS - Information Studies

Trine University

MS - Engineering Management

Nottingham Trent University

BTech - Mechanical Engineering

Vellore Institute of Technology

Similar Profiles

CREATE PROFILE