Digvijay Waghela
Dallas,TX
Summary
Data Architect with 10 years of international experience in software development, data engineering, big data, and AWS Cloud. Specialization in developing and implementing scalable, reliable, and secure data architectures to enable organizations to make informed decisions based on data assets. Experience spans multiple domains, including e-commerce, retail, and fintech.
Overview
11
11
years of professional experience
1
1
Certification
Work History
Sr. Data Engineer/Architect
Chewy
04.2023 - Current
- Led creation of Single Source of Truth (SSOT) for contacts data, improving data accuracy by 73% and serving as centralized repository for 15+ reports
- Supported Data Science team's call routing and agent recommendation initiatives, resulting in 24% improvement in accuracy and enhanced agent performance.
Data Engineer
Amazon Web Services
12.2017 - 03.2023
- Architected end-to-end data ingestion and processing pipeline using AWS Glue, Athena, and Amazon S3, resulting in 30% cost reduction and enhanced data accuracy
- Implemented Data Mesh solution for fintech customer, leveraging AWS LakeFormation, Lambda, SQS, and S3, enabling 50+ teams to centralize, own, and govern their data domains reducing complexity and saving $4.2M in licensing costs
- Led the design and implementation of scalable data analytics platform on AWS, reducing data processing time by 50% and improving overall performance
- Collaborated with stakeholders to identify business requirements and translated them into scalable and secure AWS solutions, enabling data-driven decision-making across organization
- Optimized data warehouse schema design and implemented efficient ETL processes using Amazon Redshift, resulting in 40% reduction in query response time
- Developed automated monitoring and alerting mechanisms using AWS CloudWatch and AWS Lambda, improving system reliability, and reducing downtime by 20%.
Big Data Engineer
Walmart Labs, Orzota
07.2017 - 12.2017
- Established robust 10-node Kafka Connect Cluster, ensuring high availability and scalability
- Implemented real-time data analysis pipeline using Kafka Connect, reducing data synchronization time from daily to near real-time for analytics
- Developed Source Connector for data extraction from existing sources and subsequent insertion into Kafka topics, enabling seamless and reliable data transfer
- Achieved 35% improvement in data extraction speed
- Generated real-time reports that helped business to catch spam orders worth $2.6M during Thanksgiving.
Big Data Developer
Orzota Inc
01.2017 - 07.2017
- Orchestrated OCR Document Analysis using Scala, Leptonica, Tesseract, and Ghost Script
- Converted PDFs to text files, implemented Spark script for data formatting, and stored it in HBase for subsequent analysis
- Conducted file classification and data insertion into HBase using Spark Scala, configured Solr collections, and generated dashboards in Hue Search for data visualization.
Developer & Analyst
Leap Doctor
01.2016 - 12.2016
- Executed data migration from databases like MongoDB and SQL to Amazon Redshift, demonstrating proficiency in cross-platform data transfer
- Used Pentaho for data integration, creating snowflake structure data warehouse to optimize data storage and accessibility
- Conducted ETL operations and analyzed resultant data for pattern discovery and supervised learning, showcasing expertise in data processing and machine learning
- Incorporated Rapid Prototyping and Agile methodologies in development processes, enhancing project efficiency and adaptability.
Big Data Developer
Tata Consultancy Services
11.2012 - 07.2015
- Engaged in creation of 45+ Hive tables, leading data loading and analysis via Hive queries, demonstrating proficiency in database management
- Operated with diverse file formats, including Avro, Parquet and vetted read write performance of each of these formats
Education
Master of Science - Management Information Systems
University of South Florida
Tampa, FL12.2016
Bachelor of Engineering - Electronics & Instrumentation
Shri Vaishnav Institute of Technology And Science
MP05.2012
Skills
- Experienced Data Architect with 10 years of expertise in software development, data engineering, big data
- And AWS Cloud Skilled in designing scalable and secure data architectures to empower data-driven decision- m making across e-commerce, retail, and financial domains
- Cloud & AWS Services:
- Big Data Technologies:
- Databases:
- Programming Languages:
- BI/Analytical Tools:
- Additional Tools:
- TECHNICAL SKILLS
- Glue, Lake Formation, Athena, EMR, Data Pipeline, DynamoDB, Kinesis, Lambda
- Hadoop, HDFS, Spark, Databricks, Hive, Sqoop, DBT, MapReduce, Kafka
Certification
Databricks Certified Associate Developer for Apache Spark 3.0
AWS Big Data Specialty Certification
AWS Certified Developer Associate
AWS Certified Solution Architect Associate
Cloudera CCA Spark and Hadoop Developer (CCA 175)
Timeline
Sr. Data Engineer/Architect
Chewy
04.2023 - Current
Data Engineer
Amazon Web Services
12.2017 - 03.2023
Big Data Engineer
Walmart Labs, Orzota
07.2017 - 12.2017
Big Data Developer
Orzota Inc
01.2017 - 07.2017
Developer & Analyst
Leap Doctor
01.2016 - 12.2016
Big Data Developer
Tata Consultancy Services
11.2012 - 07.2015
Master of Science - Management Information Systems
University of South Florida
Bachelor of Engineering - Electronics & Instrumentation
Shri Vaishnav Institute of Technology And Science