
Accomplished Lead Data Engineer with a proven track record at LTIMindtree, enhancing data accuracy and efficiency by 30% through innovative ETL development and performance tuning. Expert in relational databases and data modeling, with exceptional problem-solving aptitude and teamwork skills. Prioritizes seamless collaboration and technology leadership to drive project success.
• Develop a comprehensive data migration strategy in collaboration with stakeholders
• Define project timelines, milestones, and deliverables
• Allocate resources effectively to meet project goals
• Design data replication from Oracle to Snowflake using HVR
• Architect end-to-end solutions for ETL processes using DBT to transform data from CDC to Raw and Raw to Curated layer
• Develop DBT models for complex data transformation
• Reading data from CSV, JSON (Kafka topic) to base tables
• Employed advanced Snowflake features, including Snow SQL, Snow pipe, Streams, Tasks
• Designed automated processes using Python to test code accuracy and maintain coding standards.
• Identify, analyze, and troubleshoot defects, documenting and tracking issues to ensure timely resolution.
• Create data security measures are in place throughout the migration process
• Create a hashing mechanism for data security for the Snowflake table
• Implement encryption and access controls as per organizational policies
• Prepared and maintained MD70 documentation
• Source to Target mapping and functional specification
• Generate regular progress reports for stakeholders
● Led a team of 5 Data Engineers for Extract, Transform & Load (ETL) development via Informatica and Redpoint Data Management (Primary)
● Streamline data integration processes by building reusable components.
o Result: Designed and built reusable components in Informatica and RPDM for golden record implementations, reducing development time by 30%
● Led the Data team for seamless and efficient data migration from traditional RDBMS systems to Snowflake cloud data warehouse, ensuring data integrity, data governance, accuracy, data retention and optimal performance.
● Employed advanced Snowflake features, including Snow SQL, Snow pipe, Streams, Tasks and Snow Pipe to orchestrate CI/CD methodologies and optimize performance.
● Deliver value-added support in staff recruitment, selection, and training while onboarding top talent.
● Designed automated process using Python to test code accuracy and maintain coding standards.
Snowflake Migration
Optimized data pipelines by implementing advanced ETL processes and streamlining data flow.
Enhanced system performance by designing and implementing scalable data solutions for high-traffic applications.
● Lead team of 6 Snowflake developers for end-to-end data integration process using ETL and Snowflake for Structured, Semi-Structured and Unstructured data.
● Developed and optimized complex SQL queries to extract data from various sources into Snowflake, improving data accuracy and reliability.
● Leveraged Snowflake native capabilities such as Snowpipe, Snow Stream, and Snow Tasks to streamline data ingestion for
near real time data processing.
● Designed automated process using Python to test code accuracy and maintain coding standards.
Informatica Migration:
● Architected and developed data hub solution for re-insurance clients using Informatica power center by scheduling in Autosys.
● Architected solution for automating data acquisition and optimizing data delivery.
● Worked closely with other stakeholders for working on Data Security and Data Governance
Performance Tuning:
● Utilized ETL partitioning and bulk loading techniques to improve Fact table data generation performance.
● Coordination with underwriters, actuaries spread across the globe for requirement gathering and resolving issues.
● Successfully migrated all ETL jobs to Informatica, resulting in savings of over $100K on BODS, SSIS licensing, and resource costs.
● Define the scope, high-level design, architecture, project plan to lead and deliver.
· Accomplished senior developer with extensive expertise in Informatica PowerCenter, spanning over last 8-10 years. Proficient in designing, developing, and optimizing ETL workflows to ensure seamless data integration and transformation.
· Adept at crafting complex data integration solutions using Informatica PowerCenter, demonstrating a strong command of mapping, workflow design, and performance tuning techniques.
· Proven track record of successfully leading ETL projects from conception to implementation, leveraging Informatica PowerCenter's capabilities to enhance data accuracy and streamline business processes.
· Recognized for a deep understanding of data warehousing principles and best practices, consistently delivering innovative solutions that drive efficiency and maximize the value of data assets using Informatica PowerCenter.
· Along with Informatica PowerCenter, worked on Pentaho Data Integration ETL tool, SSIS ETL tools.
· Lead the team for migration WTX (IBM tool) ETL tool to Informatica Power center.
Worked on Pentaho Data Integration for Customer Data Platform (CDP) platform.