Anirudh Pallerla
Seattle,WA
Summary
- Having around 3 years of Data engineering Experience including cloud (AWS, Azure, GCP) and on-prem clusters.
- Good Knowledge of both ETL and ELT Ingestion frameworks.
- Experience in both Batch and Streaming data using Spark & Sqoop.
- Worked in both Migration projects like on-prem to Cloud and new initiatives like building features from scratch.
- Good knowledge of translating Business requirements to SQL Queries and building insights.
- Experience with ingesting RDBMS systems like MySQL, Postgres, and Oracle using Sqoop and Spark JDBC.
- Good knowledge of GIT flow and branching Strategy.
- Experience with Agile process and worked in both Scrum and Kanban methodologies.
- Experience in Different ingestion strategies like Full Refresh, Incremental and SCD Type2.
Overview
4
4
years of professional experience
Work History
Data Engineer
TCS
01.2023 - Current
- Implemented multiple AWS Lambda functions to get the user inputs from UI & to zip the reports to send back to UI.
- Creating AWS Glue scripts using Python and PySpark to transform and load the data and migrate data from Teradata to AWS RDS using SQL.
- Utilized AWS CLI to automate backups of ephemeral data stores to S3 buckets, and EBS and create nightly AMIs for mission-critical production servers as backups.
- Extensive experience in configuring Amazon EC2, Amazon S3, Amazon Elastic Load Balancing AM, and Security Groups in Public and Private Subnets in VPC and other services in the AWS Managed network security using Load balancer, and Auto-scaling. Security groups and NACL.
Data Engineer
Newton Classroom
01.2020 - 07.2021
- Developed solutions in Databricks for Data Extraction, transformation, and aggregation from multiple data sources to implement highly performant data ingestion pipelines using Azure Data Factory and Azure Databricks.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
- Integrated Spark applications using PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
Education
Master of Science - Engineering Data Science
University of Houston
Houston, TX12.2022
Bachelor of Technology - Electrical, Electronics And Communications Engineering
Jawaharlal Nehru Technological University
Hyderabad08.2020
Skills
- Python
Timeline
Data Engineer
TCS
01.2023 - Current
Data Engineer
Newton Classroom
01.2020 - 07.2021
Master of Science - Engineering Data Science
University of Houston
Bachelor of Technology - Electrical, Electronics And Communications Engineering
Jawaharlal Nehru Technological University