Anil Tadi
Milpitas,CA
Summary
A results-driven Data Engineering professional with over 14 years of experience in designing and implementing scalable data solutions across cloud and on-premises environments. Proven expertise in Snowflake, AWS, Airflow, and advanced ETL/ELT processes to drive data strategy, integration, and transformation initiatives. Skilled in optimizing performance, building robust pipelines, and enabling data-driven decision-making through efficient architectures.
Overview
15
15
years of professional experience
Work History
Senior Associate
Triniti Corporation
12.2013 - Current
- Conducted a detailed analysis of the existing data-loading application to identify bottlenecks and inefficiencies
- Analyzed data flows, column mappings, and audit trace files in the current application to document business rules and data transformation logic
- Led the redevelopment of the data-loading application, transitioning to a cloud-native, scalable solution built using Airflow for orchestration and Python for data transformations
- Designed and implemented dynamic ETL/ELT pipelines using Airflow to manage data workflows and integrate data into Snowflake
- Developed optimized queries and procedures for Snowflake, utilizing features like streams, tasks, and time travel to enhance performance and reliability
- Created automated workflows using Airflow to streamline data ingestion and transformation processes
- Managed development sprints using Agile practices, including regular stand-ups, sprint reviews, and retrospectives
- Worked closely with data engineering, QA, and cloud operations teams to oversee change control processes, conduct testing, and manage deployment activities in Snowflake and AWS
Data Engineer
Triniti Corporation
04.2022 - 06.2023
- Designed and implemented robust data extraction modules from diverse semiconductor manufacturing systems
- Applied advanced business logic transformations to semiconductor operational data to deliver insights into yield management, defect analysis, and production efficiency
- Integrated and stored processed semiconductor data into a scalable AWS Data Lake using S3 and Snowflake
- Automated data ingestion and transformation workflows using Airflow
- Built scalable data platforms on AWS and Snowflake, focusing on performance tuning, cost optimization, and advanced data modeling
- Established data governance practices with business stakeholders to ensure data integrity, security, and compliance with regulatory standards
- Translated complex business challenges into data-driven solutions using Snowflake and Airflow for big data processing
- Built and optimized self-serve ETL frameworks and streaming pipelines using Spark and Airflow
- Proactively monitored ETL pipeline performance on Snowflake and AWS, addressing bottlenecks and failures in real time
- Developed and enforced data security measures within Snowflake to protect sensitive semiconductor manufacturing data
Design/ Architect Web Application
Triniti Corporation
02.2022 - 04.2022
- Lead the requirement gathering calls, solution design and document
- Gather all the information needed to develop a New Application to create and manage all kinds of Accounts
- Collaborate with multiple teams to gather the basic data from all sources to understand data and how it can be reconciled
- Continuous monitoring the development status of the application and give an update on weekly basis
- Document the process flow like development, testing and migration activities
- Design the web application with columns/data fields displayed in different pages of the App
- Create a ETL standard architecture to populate the data into staging area from all the sources
Datawarehouse Design and Develop
Global Hotel Alliance
06.2019 - 05.2020
- Built ETL’s using SQL, PLSQL for multiple sources like flat files and DB Link making the baseline from legacy system to implement the business rules
- Minimized the load time from 2.5 days to 7.5 hours
- Automation of loading using programs, chains and jobs in DBMS_SCHEDULER
- Interact and co-ordinate with business analyst at GHA to make sure that we follow/apply the appropriate business rules while designing
- Developed more than 100 plsql packages which includes procedures and function which are running in production daily
- Design/build effective monitor process control system to capture/track the jobs executing status daily
- Documented loading process and use of monitor tables to check the live status of the load
ODI Developer and Support
Triniti Corporation
01.2019 - 05.2019
- Gathered ELT requirements from internal business team and designed its high level and low-level technical documents
- Involved with the ODI Integrator Development and understood Source systems attributes
- Performed source to Staging and staging to target mapping specifications and developed ETL / ELT ODI code for the same
- Created ODI Package of various complexity and automated process data flow
- Created Master/Work repositories, Physical/Logical schemas, Agents, and Context into ODI Topology Manager Tool
Systems Engineer
Tata Consultancy Services
04.2010 - 11.2013
- Worked as a support engineer to support/monitor the data loading process to resolve the issue caused any failure
- Worked on developing Mappings, Procedures, Shell scripts which are used for ETL jobs
- Conversion of few OWB mappings to BODI
- Worked on the BODI mappings creation for data loading
Education
Diploma in Electrical and Electronics Engineering -
AANM & VVRSR Polytechnic
01.2008
Skills
- Data Warehouse Concepts
- Unix Shell Scripting
- SQL
- PL/SQL
- OWB
- ODI
- SSIS
- Oracle Reports
- Python
- Oracle Apps
- Bigdata
- HIVE
- PIG
- Sqoop
- HBASE
- Training and mentoring
- Performance monitoring
Scholasticrecord
Diploma in Electrical and Electronics Engineering, 2008, AANM & VVRSR Polytechnic, Gudlavalleru, 74.2
Timeline
Data Engineer
Triniti Corporation
04.2022 - 06.2023
Design/ Architect Web Application
Triniti Corporation
02.2022 - 04.2022
Datawarehouse Design and Develop
Global Hotel Alliance
06.2019 - 05.2020
ODI Developer and Support
Triniti Corporation
01.2019 - 05.2019
Senior Associate
Triniti Corporation
12.2013 - Current
Systems Engineer
Tata Consultancy Services
04.2010 - 11.2013
Diploma in Electrical and Electronics Engineering -
AANM & VVRSR Polytechnic
Similar Profiles
Prathibha InturiPrathibha Inturi
Senior Technical Consultant at Triniti CorporationSenior Technical Consultant at Triniti Corporation
Michelle HeltemesMichelle Heltemes
RN Case Manager at Triniti Home Care And HospiceRN Case Manager at Triniti Home Care And Hospice
Narasimha MungiNarasimha Mungi
Technical Consultant at Triniti Advanced Software LabsTechnical Consultant at Triniti Advanced Software Labs
Lenny HidalgoLenny Hidalgo
Server at Ivy ParkServer at Ivy Park
Maritza FonsecaMaritza Fonseca
Prop 5 at Working Partnerships USAProp 5 at Working Partnerships USA