Santhosh Reddy A
Data Engineer
Summary
3 Years of IT Experience: Experienced Data Engineer Professional boasting a proven expertise in all facets of data engineering. Highlights include adeptness in refining processes, extracting and cleansing data, orchestrating transformations, and presenting visualizations. Proficient in Data Acquisition, Big Data Analytics, Distributed Computing, Data Analysis, Modeling, Cloud Services, Agile Methodologies, Data Pipelines, and both SQL and NoSQL databases. Demonstrated track record of converting raw data into actionable insights, facilitating strategic business decisions, and enhancing data operations for optimal efficiency and scalability. Responsive expert experienced in monitoring database performance, troubleshooting issues and optimizing database environment. Possesses strong analytical skills, excellent problem-solving abilities, and deep understanding of database technologies and systems. Equally confident working independently and collaboratively as needed and utilizing excellent communication skills. Background includes data mining, warehousing, and analytics. Proficient in machine and deep learning. Quality-driven and hardworking with excellent communication and project management skills. Organized and dependable candidate successful at managing multiple priorities with a positive attitude. Willingness to take on added responsibilities to meet team goals.
Overview
3
3
years of professional experience
5
5
years of post-secondary education
1
1
Certification
Work History
Data Engineer
AT&T
Remote, USA
12.2023 - Current
- Our project is to enrich AT&T customer experience by integrating and processing data from various data sources like customer interactions, usage patterns, transactions and logs and derive insights according to business requirements
- Specifically, we work on challenge of predicting customer churn by processing data and enabling downstream team to predict patterns and churn risks and enable personalize product offerings, thus enhancing customer engagement, mitigate churn risks and improve customer satisfaction.
- Developed, implemented and maintained data analytics protocols, standards, and documentation
- Designed and developed analytical data structures
- Designed scalable and maintainable data models to support business intelligence initiatives and reporting needs
- Implemented real-time data processing solutions for timely decision-making and improved customer experience
- Demonstrated strong organizational and time management skills while managing multiple projects
Data Engineer
Ible soft Pvt Ltd
Hyderabad, Telangana, India.
01.2020 - 07.2022
- project Ible Soft is adept in architecting Hadoop clusters and translating requirements into detailed designs
- Skilled in optimizing data pipelines using Azure DevOps CI/CD, specializing in GCP services like Data Proc and Big Query
- Proficient in GCP, AWS Glue, Redshift, and Snowflake for ETL and migration
- Familiar with Infrastructure as Code, leveraging Azure services like Data Factory and Data Warehouse
- Capable in developing ETL scripts, Spark apps, managing ELT pipelines with Airflow and DBT
- Well-versed in AWS, GCP, Apache Kafka, Nifi for real-time analysis and management, with knack for troubleshooting and mentoring team members.
Education
MS in computer information systems -
New England College
NH, USA 08.2022 - 05.2023
B. Tech in Electrical and Electronics Engineering - undefined
K L University
Vijayawada, India 07.2016 - 05.2020
Skills
Data Technologies: HDFS, Hive , Spark, MapReduce, PySpark, Kafka, Kubernetes
Programming Languages: SQL, Python, ScalaCloud Technologies: AWS, Microsoft Azure, Google Cloud, SnowflakeCloud Services: Amazon S3, AWS Glue, AWS Step Functions, Redshift, Lambda, AWS SNS-SQS, Amazon Aurora, Azure Data Lake, Azure Data Factory, Blob storage, DatabricksDatabases: Oracle 11g/10g, MySQL, DB2, MS SQL Server, HBASENoSQL Data Bases: Cassandra, Dynamo DB, HBaseETL, Visualization & Reporting: Power BIMonitoring tools: Apache Airflow, NifiVersion Control tools: Jenkins, GitCertification
Oracle cloud Infrastructure Foundations Associate
Etl Development
Proven expertise in Extract, Transform, Load (ETL) processes, adept at converting intricate raw data into organized, usable formats. Hands-on experience with ETL tools and frameworks to optimize data workflows and enhance data integrity.
Cloud Services
Skilled in AWS Services with practical experience across various offerings including EMR, Kinesis Stream, Glue, IAM, S3, AWS, Aurora, EC2, RDS, Elastic Load Balancer (ELB), DynamoDB, SNS, SQS, Lambda, CloudFormation. Additionally, proficient in utilizing Redshift Spectrum and AWS Athena query services for data retrieval from S3.
Database Expertise
Proficient in database management encompassing the establishment and upkeep of physical data structures across diverse platforms such as Oracle, Teradata, PostgreSQL, DB2, HBase, and SQL Server. Possess hands-on experience with Oracle, PostgreSQL, and MySQL databases. Demonstrated expertise in handling Parquet files, along with adept parsing and validation of JSON formats. Familiarity extends to NoSQL databases including HBase, Cassandra.
Data Migration
Have good Experience in Data Migration projects, Hands on experience in migrating on-premises ETL to AWS Cloud and experience in migrating from Oracle to Snowflake.
Azure Expertise
Practical expertise in Microsoft SQL Server accompanied by proficiency in Business Intelligence encompassing SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), and SQL Server Reporting Services (SSRS). Additionally, I am adept in utilizing Azure Cloud Technologies, such as Azure Database, Azure SQL, Azure Data Warehouse, Azure Data Factory (ADF), Azure Data Lake (ADL), and Azure Databricks.
Near Real Time Streaming
Good Experience working with Kafka and Spark Streaming for near real time data processing.
Data Warehousing
Skilled in understanding Data Warehousing Concepts for both OLTP and OLAP Systems, with hands-on experience in deploying solutions on platforms like Snowflake and Amazon Redshift. Developed and executed Snowflake data warehouse architecture to facilitate real-time analytics, prioritizing performance and scalability for efficient operations.
Data Visualization
I created visualizations, dashboards, sub-reports, conditional reports, drill-down reports, drill-through reports, and parameterized reports utilizing Power BI.
Workflow Management
Hands-on experience in setting up workflows with Apache Airflow, AWS Step Functions and Oozie. Scheduled, monitored, and managed complex workflows using Airflow. Optimized task dependencies and timing for efficient data processing pipelines.
Ci Cd Devops
Implemented DevOps methodologies by integrating a variety of tools such as Shell scripting, Git, Jenkins, and Docker into the data engineering workflow. This integration facilitated continuous integration, testing, and deployment processes, ensuring seamless deployment and version control of ETL and data pipeline codes. Additionally, CI/CD pipelines were established using Jenkins to automate the deployment of data processing applications.
Etl Testing
Extensive expertise in designing, developing, documenting, and testing ETL jobs and mappings in both Server and Parallel jobs using DataStage. Proficient in populating tables in Data Warehouse and Data marts. Skilled in tracing and cataloging data processes, transformation logic, and manual adjustments to pinpoint and address data issues effectively.
Analytical Skills
Strong analytical and problem-solving skills and the ability to follow through with projects from inception to completion.
Big Data Technologies
Proficient in crafting and building applications utilizing Big Data technologies such as HDFS, MapReduce, Kafka, Sqoop, Hive, PySpark, Spark SQL, Python, Databricks, and Airflow.
Timeline
Data Engineer
AT&T
12.2023 - Current
MS in computer information systems -
New England College
08.2022 - 05.2023
Data Engineer
Ible soft Pvt Ltd
01.2020 - 07.2022
B. Tech in Electrical and Electronics Engineering - undefined
K L University
07.2016 - 05.2020