Dharani Arumugam
Austin,US
Summary
Data Engineer with 4+ years of experience in data engineering, database development, and analytics. Strong hands-on experience with Python, SQL, Apache Spark, Airflow, and AWS, with a solid foundation in data warehousing, dimensional modeling, ETL pipelines, and data orchestration. Actively building real-time streaming pipelines and comfortable learning new technologies in fast-changing Agile environments.
Overview
10
10
years of professional experience
1
1
Certification
Work History
Real-Time Streaming Pipeline
01.2024 - Current
- - Built an end-to-end real-time data pipeline using Kafka, Spark Structured Streaming, Airflow, and Cassandra
- - Implemented event ingestion, streaming transformations, validation, and checkpointing
- - Orchestrated workflows with Airflow and deployed using Docker Compose
Database Developer / Analyst
Metalwest
02.2020 - 03.2021
- - Designed and maintained ETL pipelines and data warehouse workflows with 99% uptime
- - Built star-schema data models and datamarts to support analytics and reporting
- - Developed certified datasets for finance and sales stakeholders
- - Automated ingestion pipelines using SSIS and SQL
- - Built dashboards using SSRS and Qlik Sense, improving business insights by 30%
Database Developer
NTT DATA Services
12.2015 - 08.2018
- - Built and optimized large-scale data pipelines processing billions of records
- - Migrated legacy systems to modern PL/SQL-based architectures
- - Designed dimensional and relational data models
- - Performed SQL performance tuning using indexing and explain plans
- - Automated ETL workflows, reducing manual effort by 25%
Education
Master's - Data Science
Liverpool John Moores University
Bachelor's - Computer Science
Dr. Mahalingam College of Engineering & Technology
Skills
- Languages: Python, SQL, PL/SQL, T-SQL, Java (Core)
- Data Engineering: Apache Spark (PySpark), Apache Airflow, ETL/ELT, Data Pipelines, Datamarts
- Streaming (Project-based): Apache Kafka, Spark Structured Streaming
- Data Warehousing: Dimensional Modeling, Star & Snowflake Schema, Certified Datasets
- Databases: Oracle, SQL Server, PostgreSQL, Cassandra
- Cloud & Tools: AWS (S3, EC2, EMR, Redshift), Git/GitHub, Jupyter
- Methodologies: Agile / Scrum
Certification
Udacity Data Engineer Nanodegree | Oracle Certified Java Professional
Timeline
Real-Time Streaming Pipeline
01.2024 - Current
Database Developer / Analyst
Metalwest
02.2020 - 03.2021
Database Developer
NTT DATA Services
12.2015 - 08.2018
Bachelor's - Computer Science
Dr. Mahalingam College of Engineering & Technology
Master's - Data Science
Liverpool John Moores University