Summary

Overview

Work History

Education

Skills

Timeline

Karthik Gorrela

Fremont,CA

Summary

Accomplished Data Engineer with over 15 years of experience in ETL development and data analysis, including more than 7 years specializing in Big Data technologies. Proven expertise in end-to-end data management, including ETL/ELT pipeline design, dimensional data modeling, data quality assurance, data warehousing, data integration, and business intelligence solutions across healthcare, insurance, and retail domains. Hands-on experience with the Spark ecosystem, SQL, Python, PySpark,SAS, cloud platforms such as Azure, relational and NoSQL databases, and BI tools. Adept at collaborating with cross-functional teams to deliver scalable data solutions that drive business insights and operational efficiency. Expert at acting as a Subject Matter Expert (SME) to translate complex operational logic into high-throughput cloud environments while designing domain-driven architectures that eliminate data redundancies.

Overview

years of professional experience

Work History

Sr Data Engineer/ SME

Element technologies inc, Kaiser Permanente

Oakland, CA

06.2017 - Current

Cross-Functional Requirements Gathering: Collaborated directly with product owners and business users to define core business requirements, identify key metrics, and establish data mapping specifications for new and enhanced data pipelines, ensuring alignment with organizational objectives.
Data Mapping & Pipeline Enhancements: Worked alongside business analysts and BI engineers to create comprehensive data mappings and design documents for developing new data pipelines and enhancing legacy ETL jobs.
Multi-Source Data Ingestion: Engineered and managed complex ETL/ELT pipelines to ingest critical healthcare data, including referral and claims data, from diverse sources like EPIC CLARITY, CDW (Teradata, Oracle, SQL Server), and external APIs, spanning multiple markets.
Semi-Structured & File Processing: Developed automated solutions for handling structured and semi-structured data (Excel, CSV, flat files), and securely transferring data from SAS Grid to HDFS via SFTP processes.
Big Data Optimization: Managed and queried Hive tables (Big Data) effectively using Impala, optimizing data retrieval for analytical purposes.
API & JSON Ingestion: Built end-to-end pipelines to ingest COVID-19 data via APIs in JSON format, loading it into data marts to enable rapid metric development and reporting for key stakeholders.
Airflow & Python Orchestration: Led the SAS-to-Python migration effort, building scalable data pipelines in Python and orchestrating them in Apache Airflow, improving pipeline performance and maintainability.
Data Science Analytical Delivery: Utilized Python libraries such as Pandas, NumPy, and Matplotlib for data transformation, cleansing, and visualization in support of a data science initiative.
Predictive Modeling Support: Partnered closely with data scientists, delivering clean and reliable datasets for predicting projections, estimates for target setting, and also for filling the lag for the latest three months.
Azure Cloud Infrastructure: Extensively used Azure cloud services, including Synapse, Data Lake, and Azure Storage, for scalable data ingestion and processing on Azure Databricks.
ADF Pipeline Modernization: Migrated on-premises SQL Server data to Azure Data Lake using Azure Data Factory (ADF). Developed event-driven ADF pipelines for file ingestion, transformation, and automated archival.
Legacy Data Mart SME: Served as the primary Subject Matter Expert (SME) for a critical on-premise data mart (e.g., Teradata, Oracle, SQL Server), providing deep insights into data schemas, business logic, ETL processes, and historical data nuances.
Offshore Engineering Leadership: Translated complex on-premise data dependencies and legacy system logic into clear requirements for the offshore development team, ensuring accurate and efficient re-platforming.
Cloud Migration Delivery: Designed and oversaw the implementation of automated ETL/ELT pipelines for migrating historical and incremental data from the on-premises data mart to the new cloud environment.
Data Consistency & Governance: Developed detailed data mapping documents and transformation rules to ensure data consistency and accuracy post-migration.
Databricks Lakehouse Engineering: Processed and transformed large-scale healthcare data using Databricks notebooks (PySpark/SparkSQL), integrating curated results for BI reporting and ensuring clean data structures for Tableau dashboard developers.
Data Quality Validation Frameworks: Implemented data quality validation frameworks during the migration, verifying data integrity between on-premise and cloud environments (Databricks ecosystem).
Spark Performance Optimization: Optimized the SparkSQL queries and data loading processes on the new cloud platform, which is an improvement compared to the legacy on-premises system.

SAS Consultant

Element Technologies inc, Safeco Insurance Company

Seattle, WA

06.2016 - 06.2017

Multi-Source Ingestion & Pass-Thru SQL: Designed ETL pipelines which involves extracting data from Customer, Operator, Vehicle and Policy data from different data sources like (Teradata and SQL Server) and flat files using Pass Thru optimization techniques.
Dynamic Algorithmic Development: Developed a rate revision strategy Program which digests the different levels of data for each product by state and pulls the Coverage's dynamically and generate a Premium file for downstream process.
Business Intelligence Tool Delivery: Created multiple EGP files (tools) and Stored Process with prompts and filters for End users to accelerate localized analytical querying.
Data Warehouse Server Scheduling: Scheduled few Pipelines on monthly basis to load it to LASR Sever using data builder processes to maintain reporting uptime.
Data Mart Query Tuning: Managed data transformation and cleansing rules across relational databases including SQL Server to deliver reliable, structured datasets to analytical and actuarial teams.

Sr. Data Analyst

Element Technologies inc, Target Corporation

Minneapolis, MN

01.2016 - 06.2016

Interactive Visual Analytics: Created different types of interactive reports, i.e., tables, graphs, KPI reports, controls, containers, and all types of styles, interactions, display rules, filters, and roles in visual analytics.
Post-Implementation Impact Testing: Validating expected impacts post-implementation to verify business rule accuracy and system performance stability.
Workflow Operations Analysis: Involved in Workflow Analysis, Identifying the Task Structures, their Relationships and Synchronization process across enterprise analytics tasks.
Hadoop & Sqoop Data Ingestion: Ingested using Big data tools like Sqoop to pull Sales data from HDFS(Hadoop File System) to Teradata and built a pipeline to massage and harmonize the data and load to the data mart.
Heterogeneous Data Consolidation: Created a Pipeline of extracting, transformation and loading of data directly from different source systems like flat files, Excel, Oracle, Hive, PostgreSQL, Teradata and load to a data mart which in turn used for reports using Tableau.
Big Data BI Integration: Integrated Hive Tables(BigData) with BI tools like Tableau for interactive dashboards.
Server Publication & Self-Service Delivery: Published the reports on the Finance and Merchandising server which allowed end users to understand the data on the fly with the usage of quick filters for on demand needed information.
Technical Documentation & Advisory: Written or contribute to instructions or manuals to guide business users on data system updates and navigation.

ETL Tech Lead

Element Technologies inc, CSC

Columbia, MD

01.2013 - 11.2015

Summarized Dashboard Development: Converted the raw data to summarized dashboard reports that provide high level information of physicians participating in the PQRS program.
Unix-Based Data Extraction: Created pipelines which involves extracting Claims data and Encounters data from the database (ORACLE) on Unix environments.
PL/SQL-to-SAS Modernization: Migrated existing Process from PL/SQL to a SAS Job, significantly enhancing job control parameters.
Flow Automation & Studio Engineering: Experienced in creating, managing and automating in SAS EG flows and ETL Data Integration Jobs.
Web Portal Application Design: Developed Web Portal Applications using SAS BI tools like Stored Process, Information Map Studio, OLAP Cubes, Web Report Studio, and BI Dashboard.
System Debugging & Maintenance: Frequent debugging and implementation of reporting process to ensure zero data pipeline downtime.

Sr.IT Solutions Developer/IT Tech Lead

Element Technologies inc,Wellmark Health Insurance

Des Moines, IA

05.2010 - 12.2012

JAD Sessions & Technical Specifications: Conducted JAD sessions to gather business requirements and create Data Mapping and technical specifications documents for developing new or modifying existing ETL Programs.
Design Framework Governance: Expert in creating ETL Design, Data Mapping and Technical Specifications for the follow developers to ensure consistent code delivery.
Database Procedural Programming: Written stored procedures on different databases to optimize multi-table joins.
Automated Windows Scheduling: Scheduled SAS EG projects (.vbs) in Windows Task Scheduler and ETL Jobs on DI studio.
Large-Scale PROC SQL Processing: Extensively used SQL statements and PROC SQL for the analysis of large datasets (millions of records) and data manipulation and analysis.
Government Mandate Project Execution: Worked on Govt mandate projects like implementing ICD10 to update regional validation tables.
Unix Batch Shell Scripting: Automated the ETL processes in batch mode with shell scripting using KSH scripts.
SME Validation & Code Governance: Worked as a SME (Subject Matter Expert) for validation of various developed process while performing structured code reviews for SAS, SQL and KSH scripts.