Sohil Gogri

Machine Learning Pipeline Architecture and Enhancements

• Designed and implemented a pipeline for fetching data from various sources and making ML predictions.
• Created a library to extract search terms from search-related website visits.
• Retrieved email contents (drafts, sent, received) using Gmail API.
• Retrieved Google Docs contents using Drive API, ensuring access despite changes in Google’s canvas based rendering.
• Improved data quality by filtering irrelevant webpage HTML, enhancing ML predictions.
• Technologies: AWS MSK, AWS RDS, Pub/Sub for Gmail API.

Database Migration and Upgrades

• Migrated a column from Int to BigInt to prevent overflowing signed int value in the highest throughput table.
• Used AWS DMS for minimal downtime replication of DML queries to the new table.
• Ensured data integrity and continued service availability.

Infrastructure as Code (IAC) with Terraform

• Experience writing IAC using Terraform for AWS services.

On-Call Experience

• Served on-call duty for 5 years.
• Set best practices for using PagerDuty and Datadog.

AWS Cost Management

• Led initiatives for RI and savings plans to optimize EC2, RDS, S3, and Redis usage and cost.
• Reduced EC2 costs by 40% through spot fleet usage and 30% with savings plans.
• Reduced RDS/Redis costs by 33% with savings plans and S3 costs by 25% with intelligent tiering.

Monitor and Maintain High-Throughput Student Browsing Data Pipeline

• Developed pipeline for real-time storage and retrieval of student browsing data using Singlestore.
• Handles 60k requests per second at peak, with 86% updates and 14% new data.
• Maintained similar pipelines for backup of raw and deduplicated student browsing data.
• Technologies: Singlestore, AWS Kinesis streams, Spark/Scala on AWS EMR, S3.
• Critical data source for 3 out of 5 products, with 1 product reliant solely on this pipeline.

Golang Microservices Migration and Development

• Migrated 130 Golang microservices to use Go modules for service dependencies.
• Created 10 Golang microservices serving as API layers for various use cases.