Summary
Overview
Work History
Education
Skills
Languages
Timeline
Generic

Raja Shyam Nagella

New York

Summary

Senior Software Engineer – Data, with 10+ years of experience in designing, building, and optimizing large-scale data platforms. Expertise in distributed systems, data engineering, and real-time processing using technologies like Apache Kafka, Apache Iceberg, and Spark. Proven track record in developing scalable data pipelines, implementing high-performance streaming architectures, and driving data quality and governance initiatives. Strong background in cloud-based solutions (AWS) and data lakes.

Overview

12
12
years of professional experience

Work History

Senior Software Engineer - Data Platform

GrubHub
New York
05.2022 - Current
  • Developed a platform enabling self-service capabilities for streaming and batch ingestion.
  • Developed a schema registry, which stores Avro schemas and is used in self-service pipelines.
  • Implemented AWS Glue Catalog as a unified data catalog solution, following a thorough market evaluation of available catalogs, improving data discoverability and governance.
  • Implemented Apache Iceberg tooling within the Grubhub data platform through comprehensive POCs, enhancing data lake capabilities.
  • Mentored junior engineers to establish robust data ingestion pipelines from external platforms (Datadog), and provided architectural guidance on implementing scalable data quality frameworks across those pipelines.
  • Collaborated with senior engineers to define and implement Python coding standards, resulting in improved engineering excellence through enhanced best practices, maintainability, and project consistency.

Senior Data Engineer

Takeaway.com
Berlin
05.2021 - 04.2022
  • Developed a scalable, real-time courier data pipeline utilizing Kafka and Spark Streaming, replacing the batch processing system through a configuration-based approach.
  • Implemented a configuration-based PII service for Kafka events, enhancing data privacy.
  • Automated Tableau report refresh processes significantly improve product evolvability and reduce manual intervention, using the Airflow scheduler.
  • Enhanced the efficiency of courier data delivery by mentoring junior engineers on building robust, scalable data pipelines.

Associate Technical Architect

Tavant
Bangalore
11.2016 - 05.2021
  • Implemented real-time data ingestion into the data lake, utilizing Spark Structured Streaming, ensuring timely data availability for downstream processing.
  • Developed an API to automate the provisioning and management of transient and static EMR clusters across multiple versions, improving operational efficiency and scalability.
  • Created custom Azkaban plugins to automate EMR job submissions, improving workflow efficiency, and reducing manual intervention.
  • Developed data ingestion workflows to transfer Cassandra data to S3 in Parquet format, enabling optimized data access for downstream analytical applications.
  • Developed a Presto event listener for query tracking, distributed plan analysis, and data scientist query optimization using Spark Structured Streaming.

Senior Software Engineer

IBS Software Services
Trivandrum
02.2014 - 11.2016
  • Involved in consuming and converting the data (such as deck, cabin, portholes, etc.) from multiple agents using the Camel XSLT component and routing it among routes and processing it.
  • I have been involved in writing interceptors.
  • Developed the API for handling the master’s module using iBatis for the retrieval of records.

Software Engineer

CA Technologies - A Broadcom
Hyderabad
04.2013 - 01.2014
  • Interact with the key business users in order to understand the business requirements, and write documentation.
  • Responsible for applying patches on CA SDM R12.7 and validating the customer issues.
  • I have been involved in writing automation scripts.
  • Performed regression testing on CA SDM R12.9.
  • Worked with GitHub to perform version control.

Education

Bachelor of Technology - Information Technology

Santhiram Engineering College (JNTU)
Nandyal
04-2012

Skills

  • Apache Iceberg
  • PySpark
  • AWS EMR
  • AWS S3
  • Glue Catalog and Lake Formation
  • Kafka
  • Flink
  • Azkaban and Airflow
  • Pulumi

Languages

English
Professional

Timeline

Senior Software Engineer - Data Platform

GrubHub
05.2022 - Current

Senior Data Engineer

Takeaway.com
05.2021 - 04.2022

Associate Technical Architect

Tavant
11.2016 - 05.2021

Senior Software Engineer

IBS Software Services
02.2014 - 11.2016

Software Engineer

CA Technologies - A Broadcom
04.2013 - 01.2014

Bachelor of Technology - Information Technology

Santhiram Engineering College (JNTU)

Similar Profiles

CREATE PROFILE
Raja Shyam Nagella