Summary
Overview
Work History
Education
Skills
Software
Certification
Timeline
Generic

Prasad R

Data Engineer
Plano,TX

Summary

6 Years of experience in Data engineering with a comprehensive background in designing, developing, and implementing robust data solutions across diverse industries. Proven expertise in leveraging cutting-edge technologies and platforms to migrate, process, and optimize data workflows, ensuring seamless access for data consumers. Adept at orchestrating complex ETL processes, integrating various data sources, and implementing rigorous data quality measures to enhance accuracy and reliability. Skilled in fostering data governance practices, implementing security measures, and driving efficiency through automation and optimization. Proficient in a wide array of tech stacks including AWS services like red shift, Athena, S3, Jira, AWS Lambda. Also, Azure services like Azure blob storage, Azure data factory, synapse analytics, power bi, Data bricks to perform data validation, data cleansing and transforming it to data warehouse. SQL Server, with a track record of delivering impactful insights and driving business success. Detail-oriented Data engineer designs, develops and maintains highly scalable, secure and reliable data structures. To seek and maintain full-time position that offers professional challenges utilizing interpersonal skills, excellent time management and problem-solving skills.

Overview

6
6
years of professional experience
1
1
Certification

Work History

Data Engineer

JP Morgan chase
11.2022 - Current
  • Chase is a leading financial services firm, helping nearly half of America’s households and small businesses achieve their financial goals through broad range of financial products
  • I’m working on Migrating the data from Hadoop to AWS cloud by supporting multiple LOBs working on Auto, Home lending and wealth management services and to make all data available for data consumers to access
  • Responsibilities: Data is received from multiple sources like ICDW Teradata, ICDW Snowflake, info1 oracle, Analyst work bench and Manual files like SAP GL and HR Data and FRDC data sources
  • Data is ingested into the cloud using AWS transfer family MFTS and GKP Nifi leveraging SNS topics and SQS to move data to Apache Nifi hosted on EKS cluster
  • All data is stored in raw S3 bucket, and data is processed using pull-based lambda
  • Technical data quality checks are performed by triggering Nifi jobs before data is moved to trusted bucket and data that is failed in data quality checks is sent to quarantine bucket for further analysis
  • Performed business specific transformations on data in trusted bucket using Azure Databricks and moved data into refined bucket
  • The data is refined through these ETL processes which include data cleaning, validation, transformation, and enrichment before data is sent to destination
  • Integrated Databricks SQL warehouse with refined S3 bucket to query and analyze data using SQL queries directly within data bricks environment
  • Created tables and views to reference the data in refined bucket and make it available for data consumers
  • Implemented robust data security measures, we have used data masking and anonymization techniques to protect sensitive information in non-production environments and during data analytics processes
  • Leveraged Data loss prevention technologies to monitor and control flow of sensitive data within and outside the organization
  • Defined and configured python-based scripting tasks within Apache airflow DAG’s using Python operator, Bash operator and SQL operator to execute tasks and workflows efficiently
  • Managed dependencies between tasks and orchestrating complex data workflows using Apache airflow’s features like task dependencies, retries, timeouts and error handling mechanisms
  • All DAGs in Apache airflow are parameterized and metadata for tasks running in these DAG’s is present in Amazon RDS, Aurora MySQL instance
  • Extra points for 1st project Full-time Resume Utilized Jules 2.0 build for continuous integration and continuous deployment (CI/CD) pipelines for managing Apache airflow codebase, promoting changes across environments, and ensuring reproducibility and reliability of data workflows
  • Developed Egress orchestration framework to move data from Aws cloud to on-premises databases like ICDW Teradata, Essbase and FRW to make it available for data consumers to access directly from database
  • Implemented comprehensive Telemetry solutions leveraging Grafana for real-time monitoring and logging on Apache Nifi deployed on EKS cluster
  • Developed Grafana dashboards to visualize key metrics, including data flow rates facilitating actionable insights and performance optimization strategies
  • In production environment, created complex searches and reports in Splunk to analyze and correlate data across multiple data sources, identify trends, and anomalies to generate actionable insights for decision making and troubleshooting purposes
  • Tech stack: Teradata, Snowflake, AWS RDS, AWS S3, Azure Databricks, unity catalog, Apache Nifi, Apache airflow, Splunk, Grafana, AWS EKS, Jules 2.0, Python 3.12, SNS, SQS, AWS MFTS, SQL, Telemetry, Essbase, FRW, Oracle, GKP Nifi, AWS Lambda, Bitbucket, Monarch, FDLC, Jira, FDI ops portal, DSR, Hadoop, Hive, HBase, Apache pig.

Azure Data Engineer

Regions Financial corporation
08.2021 - 10.2022
  • Regions Financial Corporation is a member of the S&P 500 Index and is one of the nation’s largest full-service providers of consumer and commercial banking, wealth management and mortgage products and services
  • I have worked on extracting and analyzing large, complex data using a combination of data pipeline and analysis tools, such as SQL, Python, and Kafka
  • Experience on Migrating SQL database to Azure data Lake, and Synapse Data warehouse and controlling and granting database access and Migrating On-premises data to Azure Data Lake store using Azure migration services
  • Have extensive experience in creating pipeline jobs, scheduling triggers, Mapping data flows using Synapse pipelines
  • Experience in Developing Spark applications using Pyspark and Spark - SQL in Synapse Notebooks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns
  • Have good experience working with Azure BLOB and Data-lake storage and loading data into Azure SQL Synapse analytics (DW)
  • Perform analyses on data quality and applied business rules in all layers of data extraction transformation and loading process
  • Perform validation and verify software at all testing phases which includes Functional Testing, System Integration Testing, End to End Testing, Regression Testing, Sanity Testing, User Acceptance Testing, Smoke Testing, Disaster Recovery Testing
  • Have good experience in logging defects in Jira and Azure DevOps tools
  • Performed transformations on Azure blob storage using Azure Data Factory to filter data and this filtered data is loaded to Azure synapse analytics for other data consumers to perform analysis
  • Used Power BI to create dashboards and identify key patterns on transactional data to better serve customers
  • Created DAGs to schedule workflows using Apache Airflow
  • Developed Spark applications using PySpark and Spark – SQL in Synapse notebooks to extract, Transform and load data from multiple file formats for analyzing
  • Implemented Spark using Scala and Spark SQL for faster processing of data responsible to manage data from different sources
  • Established and enforced data governance practices, including data quality checks, metadata management, and data lineage documentation, ensuring data accuracy, consistency, and traceability across the Azure data ecosystem
  • Tech stack: Azure data lake, Azure Synapse analytics, Data Lake storage, Blob storage, Jira, PySpark, Azure data factory, power bi, Scala, Spark SQL, Python, SQL, Airflow

Database Engineer

Primera Medical Technologies
02.2020 - 07.2021
  • Primera medical technologies support and enhance care provider operations by offering comprehensive remote service with no compromise on quality
  • We are at the forefront of delivering cutting-edge, scalable technologies and strategic outsourcing solutions to hospitals and physician offices
  • I have worked on processing complexing data using Azure and Big data tools and leveraging SSRS for reporting, Power bi for identifying key performance indicators of data
  • Designed and architected scalable data processing and analytics solutions, including technical feasibility, integration, development for Big Data storage, processing and consumption of Azure data, analytics, big data (Hadoop, Spark), business intelligence (Reporting Services, Power BI), NoSQL, HDInsight, Stream Analytics, Data Factory, Event Hubs, and Notification Hubs
  • Designed end to end scalable architecture to solve business problems using various Azure Components like HDInsight, Data Factory, Data Lake, Storage and Machine Learning Studio
  • Owns the Azure technical customer engagement including architectural design sessions and implementation of projects using big data use-cases, Hadoop-based design patterns and real time/stream analytics
  • Designed an end-to-end analytical landscape involving PowerBI dashboards connected to backend SQL Server 2016 Create and manage Reports subscriptions and schedules SSRS reports
  • Designing and implementing a variety of SSRS reports such as Parameterized, Drilldown, Ad hoc and Sub-reports using Report Designer and Report Builder based on the requirements
  • Troubleshooting reports issues, ETL job failures, optimizing query performances
  • Expertise in standard Sales force configuration that include workflow rules, page layouts, Record Types, Approval Processes, Assignment rules, Validation rules, bulk Triggers, etc
  • Design and implement the ETL processes using SSIS which involves collection of data from sources like SQL Server 2008 R2/2012/2014/2016
  • Develop SSIS, SSRS / T-SQL Code and schedules jobs for Jobs Monitoring Automation
  • Implemented Event Handlers and Error Handling in SSIS packages and notified process results to various user communities
  • Created, Maintained & scheduled various reports in Power BI like Tabular Reports
  • Created effective reports using visualizations such as Bar chart, Clustered Column Chart, Waterfall Chart, Gauge, Pie Chart, Tree map etc
  • In Power BI
  • Created reconciliation report for validating migrated data
  • Tech Stack: MS SQL Server 2016/2012/2008R2, MS SQL Server Reporting Services (SSRS), MS SQL Server Integration Services (SSIS), MS SQL Server Analysis Services (SSAS), DAX, Power bi, Azure HD insight, T-SQL.

SQL/MSBI Developer

Vembu Technologies
09.2018 - 02.2020
  • Vembu is a leading provider of affordable software and cloud services
  • Vembu cloud services comprises of multiple offerings – online backup to protect files, email, databases and applications on servers and desktops, File sharing for syncing
  • Worked as a developer in creating complex Stored Procedures, Triggers, Functions, Indexes, Tables, Views and other T-SQL code and SQL joins for applications
  • Migrated data from Heterogeneous Data Sources and legacy system (DB2, Access, Excel) to SQL Server databases using SQL Server Integration Services (SSIS) to overcome transformation constraints
  • Extracted data from various sources like SQL Server 2008/2012, Oracle, .CSV, Excel and Text file from Client servers and through FTP
  • Used SSIS jobs for importing data from the flat files that bring the data to the application tables
  • Involved in loading data from various file formats (Flat file, Excel) using Oracle loader and SQL Loader
  • Created SSIS Packages using Pivot Transformation, Fuzzy Lookup, Derived Columns, Condition Split, Term extraction, Aggregate, Execute SQL Task, Data Flow Task, and Execute Package Task etc
  • To generate underlying data for the reports and to export cleaned data from Excel Spreadsheets, Text file, MS Access, and CSV files to data warehouse
  • Scheduled Cube Processing from Staging Database Tables using SQL Server Agent in SSAS
  • Gained working knowledge in Data modeling, creating star and snowflake schemas in SSAS
  • Developed complex calculated members, Named Sets using MDX expressions for specific OLAP reporting requirements
  • Designed and deployed reports with Drill Down, Drill Through, Dropdown menu option, Parameterized and Linked reports
  • Worked on deploying and scheduling Reports using SSRS and Tableau as well as triggers to generate all daily, weekly, monthly, and quarterly Reports including current status
  • Tech Stack: SQL Server 2008, SSIS, SSAS, SSRS, MS SQL Server 2012/2008/2005, Tableau 7.0/8.1 Server and Desktop, Eclipse, Windows Server/professional, Oracle 12g, C#, VISIO, ERWIN.

Education

Master of Science - Computer And Information Sciences

Wichita State University
Wichita, KS
05.2001 -

Skills

undefined

Software

Python

Java

Scala

SQL

R programming

Certification

AWS certified Data analytics

Timeline

Data Engineer

JP Morgan chase
11.2022 - Current

Azure Data Engineer

Regions Financial corporation
08.2021 - 10.2022

Database Engineer

Primera Medical Technologies
02.2020 - 07.2021

SQL/MSBI Developer

Vembu Technologies
09.2018 - 02.2020

Master of Science - Computer And Information Sciences

Wichita State University
05.2001 -

Similar Profiles

CREATE PROFILE
Prasad RData Engineer