Shilpa Singumahanthi
Data Engineer
Redmond,WA
Summary
Over 5 years of IT experience in Big Data and Business Intelligence technologies.
• Proficient in SQL, Python and Extraction, Transformation and Loading (ETL) data from various sources into Data Warehouses
• Worked on Hadoop ecosystem technologies including Hadoop, Spark, MapReduce, Kafka, Hive, Sqoop and HBase.
• Experience in analyzing data using HiveQL, Spark SQL.
• Experience with Hadoop Distributions like Cloudera and Hortonworks.
• Worked on Dimensional Data modelling in Star and Snowflake schemas and Slowly Changing Dimensions (SCD).
• Extensive experience in Amazon Redshift, managing and optimizing Amazon Redshift databases for large-scale data warehousing projects.
• Experienced in database monitoring, troubleshooting, and optimization.
• Strong analytical and problem-solving skills with excellent communication abilities.
Overview
6
6
years of professional experience
4
4
years of post-secondary education
4
4
Certifications
Work History
Sr. Data Engineer
Hilton
10.2017 - 12.2018
- Designed and implemented database schemas in Amazon Redshift to support efficient data storage, retrieval, and analytics
- Monitored and tuned Redshift clusters to ensure optimal performance and scalability, including monitoring query execution times, analyzing query plans, and identifying and resolving performance bottlenecks
- Developed and fine-tuned WLM queues to prioritize and manage different types of workloads based on their criticality and resource requirements
- Utilized S3 bucket and Glacier for storage and backup on AWS
- Designed and implemented query prioritization strategies in Amazon Redshift to optimize query execution and meet critical business requirements
- Utilized Redshift's elastic resize capabilities to scale compute and storage resources up or down based on workload demands, optimizing cost-efficiency without compromising performance
- Familiarity with Redshift automation and orchestration tools, such as AWS Glue, or custom scripting, to streamline and automate routine database administration tasks
- Using Amazon Identity Access Management (IAM) tool created groups & permissions for users to work collaboratively
- Day to-day responsibility includes developing ETL Pipelines in and out of data warehouse, develop major regulatory and financial reports using advanced SQL queries in Redshift.
Data Engineer
Stifel
03.2016 - 09.2017
- Involved in requirement gathering/analysis, Design, Development, Testing and Production rollover of Reporting and Analysis projects
- Interacting with Business, Business Analysts and source system SMEs
- Analyzed multiple source systems such as My SQL, Salesforce, DB2 to come up with ETL mapping document.
- Design data model for Sales data
- Created Rich dashboards using Tableau.
- Used excel sheet, flat files, CSV files to generate Tableau reports.
- Provided customer support to Tableau users and wrote Custom SQL to support business requirements.
Data Engineer
Tata Consultancy Services
01.2013 - 01.2016
- Preparing ETL mapping documents
- Setup ETL data pipeline using Informatica
- Communicating with Business to understand their requirements.
- Extensively involved in doing user acceptance testing (UAT).
- Data migration, identifying conflicts between different source systems and design for Master Data (Customers and Contacts).
- Data reconciliation setup between Legacy and SFDC systems
- Build Data Dictionaries by extracted the keywords from multiple data file formats such as XML, CSV
- Developing AQL code to read the files and identify keywords with the help of data dictionaries
- Created Hive tables and data pipeline setup to load data.
- Querying Hive tables to retrieve the data
- Developed test cases for Unit Testing
- Performed data validation using Splunk
- Coordinated with various members of the team to resolve defects
Education
B.Tech Information Technology -
JNTU
08.2008 - 05.2012
Skills
SQL
Python
Hadoop Stack: HDFS, MapReduce, Hive, HBase
AWS Stack: Amazon S3, Amazon Redshift, Lambda, IAM, SNS, SQS, CloudWatch, EC2, EMR, Managed Apache Airflow, RDS, AWS Glue
Spark
Oracle 9i/10g/11g/12c
Informatica Power Center
Power BI
Certification
Big Data and Hadoop certification from Edureka
Timeline
Sr. Data Engineer
Hilton
10.2017 - 12.2018
Data Engineer
Stifel
03.2016 - 09.2017
Data Engineer
Tata Consultancy Services
01.2013 - 01.2016
B.Tech Information Technology -
JNTU
08.2008 - 05.2012