Karthick Raja Selvam
Cary,NC
Summary
Senior Data Engineer with expertise in developing, testing, and maintaining data architectures. Proficient in database management systems, Big Data processing frameworks, and data modeling. Proven track record of leading teams to create innovative data solutions that optimize system efficiency and inform business decision-making. Success in improving data availability and accuracy in previous roles.
Overview
11
11
years of professional experience
Work History
Senior Data Engineer
Capgemini Technology Services
Cary, NC
09.2022 - Current
- Utilized snowflake metadata from 50 multiple accounts to construct an efficient datamart/data warehouse.
- Ensured accurate reconciliation of snowflake usage statement with metadata in data warehouse.
- Engaged in regular communication with the Snowflake support team, effectively addressing issues and seeking clarification.
- Utilized Snowpark, Pandas, and other relevant package utilities to create a highly effective end-to-end dynamic pipeline in Python.
- Implemented streamlined data modeling techniques to enhance efficiency of metadata pipeline.
- Employed data warehouse metadata to design and launch Power BI dashboards, enabling analysis of monthly snowflake consumption across a wide range of service types within the organization.
- Successfully implemented DBT proof of concept, facilitating the creation of dimension and fact tables within the Transformation and Loading layer.
- Utilized different techniques to optimize costs on the Snowflake platform, leading to a noticeable decrease in expenses.
- Generated Power BI reports to depict the analysis of cost optimization techniques.
- Established connections with other BU/Teams to access required data from datamart, ensuring adherence to data governance protocols.
Senior Data Engineer
Capgemini Technology Services
Cary, NC
12.2019 - 08.2022
- Examined source file and formulated relevant workflow
- Developed a versatile shell script to transfer files from S3 to Snowflake for various sources.
- Established a streamlined execution Workflow through implementation of the control -m tool.
- Implemented the use of Stash to enhance data organization and efficiency.
- Effectively utilized XML, JSON, CSV, and TAB file formats to import data into the snowflake staging layer.
Data Engineer
MathCo (TheMathCompany)
Cary, NC
04.2019 - 11.2019
- Utilized a Dataproc Cluster to effectively transform, process, and store billions of data in BigQuery.
- Developed data model diagram and shell scripts utilized in the execution of Hive and Spark scripts on a Dataproc cluster.
- Managed job scheduling with Airflow.
- Developed Python and PySpark scripts for data transformation and loading onto GCP Cloud Platform
Data Engineer
Sri mookambika info solutions
Cary, NC
10.2013 - 03.2019
- Performed requirement analysis, estimated project duration, and developed mapping documents for data modeling purposes.
- Utilized strong proficiency in creating PL/SQL packages, procedures, functions, triggers, and views.
- Efficiently performed data transfers from Oracle, Postgres, MySQL, and Flat files through the creation of effective ETL jobs for successful loading onto S3.
- Provided the Product team with data analysis capabilities through the creation and accessibility of a S3 Data Lake.
- Performed production deployment tasks and assisted with support responsibilities.
- Extensively used AWS services including S3, EC2, Redshift, Athena, Glue and Database Migration Services.
- Engaged in responsibilities encompassing data backup/restore procedures, database construction activities, schema organization efforts for efficient access control measures with associated users and roles.
- Collaborated with Application team to generate tables for KPI dashboards and Statement Reports.
Education
Master of Science - Computer And Information Sciences
Kalasalingam University
Cary, NC04-2012
Bachelor of Science - Computer And Information Sciences
Yadava College
Cary, NC04-2009
Skills
- Snowflake
- Pyspark
- Python Programming
- Redshift
- Data Visualization
- Analytical Skills
- Data Modeling
- Performance Tuning
- Data Warehousing
- Jenkins CI-CD
Timeline
Senior Data Engineer
Capgemini Technology Services
09.2022 - Current
Senior Data Engineer
Capgemini Technology Services
12.2019 - 08.2022
Data Engineer
MathCo (TheMathCompany)
04.2019 - 11.2019
Data Engineer
Sri mookambika info solutions
10.2013 - 03.2019
Master of Science - Computer And Information Sciences
Kalasalingam University
Bachelor of Science - Computer And Information Sciences
Yadava College
Similar Profiles
Jitendra UpadhyayJitendra Upadhyay
Design, Develop and Implementation at Capgemini Technology Services India LimitedDesign, Develop and Implementation at Capgemini Technology Services India Limited
Sourav GangulySourav Ganguly
Team Manager at Capgemini Technology Services India LimitedTeam Manager at Capgemini Technology Services India Limited
Umang GuptaUmang Gupta
Senior Analyst at Capgemini Technology Services India Ltd.Senior Analyst at Capgemini Technology Services India Ltd.
Vishal SinghVishal Singh
Manager - RTR at Capgemini Technology Services India LimitedManager - RTR at Capgemini Technology Services India Limited
Dillon MarrDillon Marr
General Manager - North Carolina at Secure Parking USAGeneral Manager - North Carolina at Secure Parking USA