Sudhakar Selvarajan

• Maintain and oversee platform engineering for all data platforms which includes Databricks, Snowflake, Informatica, Cleo, Alteryx and GCP BigData.
• Manage platforms and finances for projects worth more than $20 million.
• Lead machine learning engineering - Create and maintain MLOPS architecture, Feature engineering and venturing into LLMs and LLMOPS
• Lead data modernization projects like Data maturity program which involves decommissioning of workloads running on Informatica power center and netezza and move them to Spark on databricks. Migrating 2000 tables and 800 jobs to databricks.
• Implement Data Clean rooms and Data sharing using cloud based technologies
• Lead cross-functional teams to achieve project goals.
• Reduce operational costs through comprehensive process improvement initiatives and resource management.
• Lead innovation strategy for data organization.
• Achieved significant cost savings by renegotiating contracts with key vendors, without compromising service quality.
• Developed strong company culture focused on employee engagement, collaboration, and continuous learning opportunities - Created center of excellence and office hours for data engineering.
• Maintain and oversee vendor relations and finances for all the data platforms.

• Strong technical leadership skills with experience in managing customer, vendor and teams .
• Led a global team of 60 bigdata data engineers .
• Provided technical and architectural insights to my team to help them create Ingestion, data quality and data comparison frameworks in databricks
• Streamlined project delivery processes, significantly reducing time to market for new product launches.
• Organized professional development programs for staff, leading to improved performance and skill sets.
• Achieved departmental goals by developing and executing strategic plans and performance metrics.
• Enhanced team productivity by implementing agile methodologies, leading to more efficient project completion.
• Managed budgets effectively, ensuring optimal financial performance while investing in necessary resources for business growth.
• Built high-performing teams through effective recruitment, onboarding, and talent development initiatives.

• Led team of 5 Big Data engineers in designing, developing, and maintaining data pipelines.
• Architected end-to-end data solutions using Databricks.
• Designed and developed complex ETL pipelines using Apache Spark and Databricks,.
• Championed best practices for data engineering within organization.
• Managed performance tuning and optimization efforts for Databricks and Snowflake,.
• Mentored and provided technical guidance to junior team members.
• Evaluated and selected appropriate technologies and tools for data processing, storage, and analytics.
• Collaborated with cross-functional teams to ensure data governance, security, and compliance requirements were met throughout the data engineering lifecycle.
• Designed and maintained data architecture documentation, data flow diagrams, and technical specifications.
• Collaborated with infrastructure and operations teams to manage and scale the Databricks and Snowflake environments,.

• Designed, developed, and maintained complex data pipelines using Databricks and Apache Spark.
• Implemented performance optimizations and tuning techniques on Spark jobs.
• Led integration of real-time and batch processing pipelines within Databricks.
• Implemented data quality checks and validation processes.
• Created process of code reviews in data engineering team.
• Troubleshoot and resolved complex data issues.

Client : Dignity Health

• Led a team to build data ingestion pipelines to migrate data from legacy mainframe application to Hadoop ecosystem.
• Designed and developed complex, near real time, severe sepsis time zero and sepsis bundle compliance logic in SQOOP, PIG and Python which was displayed in dashboards and apps to help physicians make clinical decisions
• Built data pipelines to bring in population health data into Hadoop ecosystem, archive the data and send it out to Athena
• Implemented the machine learning algorithm that was provided by the clinicians, in PIG and Python
• Designed and developed near real time data pipelines to bring clinical and ADT data, using SQOOP and PIG scripts and sent them over to third party vendor to trigger alerts with respect to patient safety in hospitals

Client : American Express

• Creation of various data models for storing merchant information.
• Perform strategic data analysis and research to device predictive analysis algorithm.
• Optimize ETL process to enable faster and efficient transformation and loads.
• Migration of DB from DB2 to Netezza.
• Migration of ETL from Datastage 8.1 to 8.7.
• Created and managed financial capture applications