KAMAL TEJA MANCHU
Greenville,SC
Summary
Accomplished Analytics Engineer with a proven track record in designing, implementing, and optimizing cloud solutions with a primary focus on Microsoft Azure. Proven proficiency in data analytics, reporting, data architecture, ETL processes, and optimizing data storage for improved performance. Adept in leveraging Azure data engineering services to enhance efficiency, scalability, and reliability of big data solutions. Seeking to contribute my technical skills and knowledge to a dynamic analytics team.
Overview
7
7
years of professional experience
Work History
AI Data Engineer
AIBES
01.2024 - Current
- Directed a comprehensive initiative to enhance security infrastructure for both Active Directory and cloud services while simultaneously leading the development of data integration and reporting solutions using Azure Data Factory (ADF) and Power BI
- The project included the design and deployment of ELT pipelines from multiple data sources, the creation of impactful reports and dashboards, and the automation of financial reporting processes
- Additionally, Power BI training sessions were conducted to improve team productivity, and optimized solutions were deployed to reduce licensing costs.
Azure Data Engineer
Cognizant
09.2022 - 08.2023
- Led the successful migration of a primary application backend and data warehouse architecture from on-premise systems to the Azure cloud, significantly enhancing performance, scalability, and reliability
- This project involved designing and implementing scalable ETL pipelines in Azure Synapse, creating and optimizing metadata-driven solutions, and managing CI/CD pipelines using ARM templates to ensure smooth deployment across environments
- The project was recognized with an EXCEL award for innovative pipeline management and included mentoring junior team members to drive project success.
Azure Data Engineer
Tata Consultancy Services
05.2021 - 08.2022
- Developed and optimized comprehensive ELT pipelines in Azure Data Factory (ADF), leveraging Databricks Notebooks and Delta Lake architecture to enhance data processing efficiency and execution time
- The project focused on migrating pipelines to Delta Lake from Azure Synapse Analytics, optimizing ARM templates for improved CI/CD processes, and utilizing Python and PySpark for advanced data analysis and solution development.
Data/Machine Learning Engineer
Tata Consultancy Services
03.2020 - 04.2021
- Led the analysis, profiling, and development of ETL solutions for multiple systems using complex SQL, while directing the migration of legacy systems to Microsoft Azure
- The project involved building and optimizing data pipelines using Azure Data Factory, as well as developing and deploying machine learning models for advanced data analysis and process automation.
Data Engineer
Tata Consultancy Services
11.2017 - 02.2020
- Installed and configured Hadoop Map Reduce, HDFS, developed multiple Map Reduce jobs in java for data cleaning and pre-processing
- Gathered requirements for ingestion of new data sources including life cycle, data quality check, transformations, and metadata enrichment
- Created data pipelines for different events to load the data from DynamoDB to AWS S3 bucket and then into HDFS and delivered high success metrics
- Importing data from MS SQL server and Teradata into HDFS using Sqoop
- Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files
- Implemented data streaming capability using Kafka for multiple data sources
- Worked with multiple storage formats (Avro, Parquet) and databases (Hive, Impala, Kudu)
- Worked on connecting Cassandra database to the Amazon EMR File System for storing the database in S3
- Created Oozie workflows to automate and productionize the data pipelines
Education
MS in Business Analytics -
University of Louisville
Louisville, KYPost Graduation Diploma in Data Science -
IIIT Bangalore
Bangalore, IndiaSkills
- Azure Cloud Services
- Azure Synapse Analytics
- Azure Data Factory
- Azure Data Lake Services
- Azure Blob Storage
- Azure Databricks
- Azure Devops
- Azure Keyvault
- Azure Machine Learning
- Azure Stream Analytics
- PowerShell scripting
- Azure DevOps
- Azure Resource Manager (ARM)
- Jira
- ServiceNow
- VB
- TSQL
- Python
- R
- Spark
- ETL Processes
- Database Optimization
- Query Optimization
- Data Transformation
- Regression
- Classification
- Clustering
- Scikit Learn
- Statsmodel
- Hugging face
- Tensorflow
- Matplotlib
- Seaborn
- TF-IDF
- LLM
- AutoGen
- DevOps practices
- Database management
- Leadership skills
- Python & SQL Programming
- CI/CD Pipelines
- Mentorship & Leadership
- Security and compliance
- Scripting and automation
- Troubleshooting/problem solving
- Data Lake Storage
- Machine Learning (ML)
- Databricks
- PySpark
- Azure cloud services
- Collaboration and communication
- ETL Process
- Data Architecture & Modeling
- Metadata-Driven Pipelines
- Power BI, Tableau
- System design and architecture
- Teamwork
- Financial acumen
- Data Visualization
- ETL processes
- Python Programming
- Database Management
- Microsoft SQL Server
- Data Modeling
- ETL development
- Data Warehousing
- Big Data Processing
- Data Pipeline Design
- Machine Learning
- Spark Framework
Personal Information
Title: Data Engineer
Timeline
AI Data Engineer
AIBES
01.2024 - Current
Azure Data Engineer
Cognizant
09.2022 - 08.2023
Azure Data Engineer
Tata Consultancy Services
05.2021 - 08.2022
Data/Machine Learning Engineer
Tata Consultancy Services
03.2020 - 04.2021
Data Engineer
Tata Consultancy Services
11.2017 - 02.2020
MS in Business Analytics -
University of Louisville
Post Graduation Diploma in Data Science -
IIIT Bangalore
Similar Profiles
Bradley CareyBradley Carey
Carpet Cleaner at Qualicare Commercial CleaningCarpet Cleaner at Qualicare Commercial Cleaning
John LockeJohn Locke
Store Manager at Gotta RunStore Manager at Gotta Run
Harold ClevelandHarold Cleveland
Remote Claim Processor at CopartRemote Claim Processor at Copart
Raina PatelRaina Patel
Volunteer at Ronald McDonald House Charities, IncVolunteer at Ronald McDonald House Charities, Inc
SONIA SATIJASONIA SATIJA
Data Engineer/ ML AI Data Engineer at IBM INDIA PVT. LTDData Engineer/ ML AI Data Engineer at IBM INDIA PVT. LTD