Summary
Overview
Work History
Education
Skills
Projects
Timeline
Generic

Krishna Kishore Gadiyamula

Irving,TX

Summary

Driven Data Engineer with a proven track record at FedEx, where I spearheaded the migration of SQL databases to Azure, enhancing Spark performance and automating ETL workflows, resulting in a 50% reduction in manual effort. Expert in Python, Apache Airflow, and data governance, I excel in transforming data into actionable insights and secure, efficient data solutions.

Overview

5
5
years of professional experience

Work History

Data Engineer

Splunk
San Francisco, CA
08.2023 - Current
  • Built Spark-based pipelines for processing large-scale log data, improving performance by 40%
  • Developed and automated ETL workflows using Python, Apache Airflow, and Azure Data Factory
  • Integrated data into Azure Synapse and designed optimized storage strategies using Delta Lake
  • Improved real-time monitoring with Kafka and implemented role-based access control for secure data solutions

Data Engineer

FedEx
Memphis, TN
05.2018 - 07.2022
  • Migrated enterprise SQL databases to the Azure ecosystem, including Azure Data Lake and Azure SQL Database
  • Enhanced Spark performance by implementing partitioning, caching, and other tuning techniques
  • Designed and deployed Apache Kafka pipelines for real-time data ingestion from diverse sources
  • Developed Power BI dashboards to provide actionable insights into business metrics
  • Automated ETL workflows with Apache Airflow, reducing manual effort by 50%

Education

Master of Science - Computer Science

Wright State University
Dayton, OH
05.2024

Skills

  • Python
  • PySpark
  • Scala
  • Java
  • SQL
  • Bash Scripting
  • Bash
  • JSON
  • YAML
  • Apache Spark
  • Hadoop Ecosystem
  • Kafka
  • Flink
  • Data Flow Automation with Nifi
  • Presto
  • Impala
  • AWS
  • Azure
  • Google Cloud Platform
  • Snowflake
  • Amazon Redshift
  • Teradata Database Management
  • Informatica PowerCenter
  • SSIS
  • Apache Airflow
  • Talend
  • Matillion Data Integration Specialist
  • Jenkins
  • Docker
  • Kubernetes
  • Terraform
  • Ansible Configuration Management
  • Git
  • GitHub Actions
  • Azure DevOps
  • Airflow DAGs
  • Informatica IDQ
  • Apache Atlas
  • Collibra Proficiency
  • Data Governance Expertise
  • Data profiling
  • Metadata Management
  • DynamoDB
  • MongoDB
  • Cassandra
  • HBase Database Management
  • Couchbase
  • PostgreSQL
  • MySQL
  • Oracle
  • Microsoft SQL Server
  • Power BI
  • Tableau
  • QlikView Expertise
  • Looker
  • Superset
  • ELK Stack
  • Apache Hudi Proficiency
  • Apache Iceberg
  • Delta Lake
  • Cloudera
  • Hortonworks Data Solutions
  • Data modeling
  • Data Lake Architecture
  • Streaming analytics
  • Live Data Analysis
  • Batch Data Pipelines
  • Data partitioning
  • Caching
  • Optimization
  • TensorFlow
  • PyTorch
  • Scikit-learn
  • Bitbucket
  • JIRA
  • Confluence
  • ETL development
  • Machine learning
  • Data integration
  • NoSQL databases
  • Data curating
  • Real-time analytics
  • Data pipeline design
  • Big data processing
  • Data governance
  • Data security
  • Performance tuning
  • Spark framework
  • Data warehousing
  • Hadoop ecosystem
  • API development
  • Data quality assurance
  • Data pipeline control
  • SQL expertise
  • Metadata management
  • Scripting languages
  • Data migration

Projects

Streaming Analytics Framework, Designed and implemented a real-time streaming solution with Kafka and Spark Streaming for event-driven analytics. AWS Data Lake Migration, Built a secure AWS S3-based data lake, integrating with Redshift and Athena for high-performance analytics. Interactive Dashboards, Developed Power BI dashboards to visualize data from Snowflake and Azure Synapse, enhancing decision-making.

Timeline

Data Engineer

Splunk
08.2023 - Current

Data Engineer

FedEx
05.2018 - 07.2022

Master of Science - Computer Science

Wright State University

Similar Profiles

CREATE PROFILE
Krishna Kishore Gadiyamula