Lead Data Engineer
- Designed and implemented data engineering and analytics solutions to enable Walgreens Health Data and Insight commercialization and enable automation of payer data metrics as part of care of quality reporting.
- Leveraged Azure Data Factory pipelines, Databricks using Pyspark, and Snowflake SQL to build scalable data infrastructure for automated data ingestion and processing leading to a 30% increase in data processing capabilities for the Insights platform.
- Developed and maintained data integration processes across Azure and Snowflake thereby making it easier for cross functional teams to collaborate on data ingestion processes for different sources.
- Streamlined data ingestion processes to accommodate increasing volumes of incoming information while maintaining data integrity and ensuring timely accessibility across the organization.
- Designed and implemented data security and access control policies for Azure and Snowflake
- Optimized data processing by implementing cutting-edge data engineering techniques and tools.
- Identified areas for improvement within existing ETL frameworks to maximize efficiency without sacrificing accuracy or integrity of results produced during transformations applied on ingested datasets.
- Increased efficiency of ETL processes by automating manual tasks, reducing processing time and resource consumption.