Rajat Ahuja
San Francisco,United States
Summary
To gain employment with an organization/corporation of international repute where my technical and managerial skills can be used and honed effectively.
Overview
11
11
years of professional experience
Work History
Senior Software Engineer
Twitter
San Francisco, CA
05.2018 - Current
- Collaborated on various Data Processing Libraries like Scalding, Spark, Dataflow to run an End to End job in the most simplistic manner at Twitter.
- Integrated Scalding/Spark/Dataflow jobs with Twitter Data Discovery and State-managed Services to run Scheduled jobs on top of Kubernetes/Aurora.
- Build an Auto-tuning Service for Scalding Jobs to dynamically adjust the Containers Memory/Numbers necessary for the Job depending on the history of previous scheduled runs, resulting in greater Hadoop Cluster usage.
- Lead a project to migrate existing on-premises infrastructure to Google Cloud Platform in order to perform batch jobs.
- Collaborated with peers to develop an Event-Driven Scheduler for replicating data from various Data stores, including Hadoop distributed file system, Google Cloud Storage, Key Value NoSQL, BigQuery.
- Develop a Managed Service to replicate data from BigQuery to our in-house Key-value store in order to serve machine learning models in the Serving path.
- Consolidated all backend metrics into a single store that is used for visualization and to automatically optimize scala/spark jobs.
Tech Lead
InMobi
Bangalore, India
04.2012 - 05.2018
- Implemented a new Spark Join API (Bucked By Bucket Join) that joins Map/Reduced Partitioned Datasets based on the available partitions.
It decreased overall duration of some crucial jobs by 80 %.This idea was inspired by Map Side join in Map reduce. - Build pipelines to measure how many people near the point of interest (Macdonald's, Universities) were seen in a certain time frame (hour, day, week) and within a certain radius (250m, 500m)
- Collaborated on the creation of services that locate the user's location based on Lat Long, IP to calculate the Zip/City/State to efficiently target advertisements.
- Built Geo-based location-hygiene pipelines to eliminate fraud Geo-inaccurate site IDs.
Education
Information Technology
Indian Institute of Information Technology
Allahabad06.2011
Skills
- Java, Scala, Python
- Mysql, Redis, Cassandra, Key/Value NoSql
- Scalding, Spark, Dataflow, Beam, Map Reduce
- Google Cloud Platform, HDFS, Yarn, Thirft RPC,
Timeline
Senior Software Engineer
Twitter
05.2018 - Current
Tech Lead
InMobi
04.2012 - 05.2018
Information Technology
Indian Institute of Information Technology