Divya Ganesan
Ashburn,VA
Summary
Seasoned Data Engineer with over 18 years of experience, specializing in data engineering, cloud migration and ETL Processes.
Overview
18
18
years of professional experience
1
1
Certification
Work History
Lead Engineer
BOOZ ALLEN HAMILTON
11.2022 - 06.2024
- Designed a robust pipeline to extract data from the silver layer, utilizing Spark in Databricks to construct JSON
- Managed both full and incremental loads, pushing updates to the RDS PostgreSQL database for API consumption
- Implemented an Audit Trail system to capture and store auditable data using Databricks, facilitating faster data search and analysis
- Acted as the primary point of contact for production issues and spearheaded the implementation of new enhancements and requirements for the system
- Communicated with business users to understand requirements and articulate the impact on systems, providing various implementation suggestions
- Provided troubleshooting and debugging support for day-to-day production issues, collaborating with other infrastructure teams as necessary.
Data Engineering Manager
IBM
03.2012 - 11.2022
- As the Data Engineer Lead, collaborated with Client PMO, legacy system owners, and external interfacing partners on requirements, design, implementation, and resolving complex data merge issues
- Successfully led the LSAM Data team for the migration from DUNS Number to the Unique Entity ID used across the federal government
- Managed quarterly releases including builds, software fixes, and technological upgrades for database technologies like Oracle, Postgres, Cloveretl JDK, and JBoss EAP
- Hands-on experience with Oracle to Postgres database migration using ora2pg and foreign wrappers (FDW) for Microservices-based architecture
- Migrated all data and pipelines to AWS from on-prem (Data Warehouse)
- Key resource for identifying fraudulent activity in SAM.gov and providing user audit reports to the Office of Inspector General
- Developed the API tracker and coordinated releases supporting both Agile and Waterfall methodologies
- Supported functional qualification testing for deliverables and analyzed/resolved system integration problems
- Managed and performed production deployment activities and supported operations and maintenance
- Refactored critical pipelines involving Entity, Exclusions, Webservices, IRS, CAGE, DNB, FMS (Treasury), CEFT, MOCAS, and supported numerous ad hoc requests.
Senior ETL Developer
FINRA
02.2011 - 03.2012
- Analyzed business requirements, performed source system analysis, and prepared technical design documents and source-to-target data mapping documents
- Developed ETL processes to populate the Enterprise Product Orders data mart dimensions and facts using Informatica, Oracle SQL, and UNIX shell scripts
- Developed and scheduled jobs for the daily load of the Orders data mart, invoking Informatica workflows and sessions, SQL scripts for index management, and Unix scripts for parameter file updates and database connectivity checks
- Reviewed code, design, and test plans throughout the project lifecycle.
ETL Developer
Scope International Bank
09.2006 - 10.2011
- Designed and maintained logical and physical schemas using ERWIN
- Consolidated data from various departments, such as Mortgage, into the data warehouse by identifying common data and resolving discrepancies, creating various data marts
- Developed PL/SQL procedures for processing business logic in the database
- Scheduled Informatica workflows in Autosys to automate update processes
- Scheduled reports to run on a monthly, weekly, and daily basis using Cognos Connection Scheduler.
Education
BS - Computer Science and Engineering
ADU tech, India
Skills
Databases
- PostgreSQL
- Oracle 9i/10g/11g
- DB2
- PL/SQL
ETL Tools
- Databricks
- AWS Glue
- Athena
- Informatica Power Center 86
- Clover 30-55
- IBM Datastage
Working knowledge of
- AWS
- Python
- Linux
- UNIX Shell Programming
- Teradata
- Terraform
- Splunk
- HTML
- Tableau
- Business Objects XI R3/R2
- IBM Cognos
Certification
- Databricks Data Engineer Associate
- Dev-ops practitioner
Clearance
IRS Public Trust Security Clearance
Timeline
Lead Engineer
BOOZ ALLEN HAMILTON
11.2022 - 06.2024
Data Engineering Manager
IBM
03.2012 - 11.2022
Senior ETL Developer
FINRA
02.2011 - 03.2012
ETL Developer
Scope International Bank
09.2006 - 10.2011
BS - Computer Science and Engineering
ADU tech, India