As a founding engineer, built the Log Management product from the ground up - Ingestion, Parsing, Storage engine and Query layer. (https://www.kloudfuse.com/blog/highcardinality-blog-series-part-4)
Contributed heavily to Apache Pinot internally (https://github.com/apache/pinot). Some of these contributions are open source as well.
Scaled the product to be able to do 1.5 million logs per second and query more than 100 billion logs on a single cluster..
Logs product helped the company acquire large customers in the US and India generating millions in revenue.
Senior Software Engineer
AWS
Palo Alto, California
08.2020 - 02.2021
Contributed to improve observability and troubleshooting pipeline for the team.
Wrote some tools to analyze incoming logs and metric signals and point potential issues with customer deployments
Was with AWS for a very short period of time
Senior Software Engineer
Autonomic.ai
Palo Alto, California
05.2018 - 08.2020
Lead the company's effort to build a scalable core platform that manages an inventory of millions of assets (Connected Vehicles and Devices), and also lead the effort to make the platform multi-tenant.
Contributions brought down API latencies from over 100ms to single digit milliseconds at throughputs around 50k ops
Segregated a core service into multiple scalable services with different specific responsibilities - Search, Audit history, Source of truth etc
Senior Software Engineer
Springpath Inc
Sunnyvale, California
01.2017 - 05.2018
Designed and implemented Springpath's cluster to cluster data replication over WAN using C++ and Python
Implemented several optimizations like connection caching, asynchronous restartability of replications
Design techniques for WAN optimization/deduplication for replication to reduce network traffic, faster replications and more efficient disk usage
Software Engineer
Axcient Inc
Mountain View, California
12.2013 - 01.2017
Major contribution towards building and delivering Axcient Fusion to market
Designed and implemented deduplicating data replicator for Axcient Fusion using merkle-trees for intelligent deduplication. Optimized replication and recovery for better RPO and RTO by finding the exact list of used and unallocated extents on disk.
Contributions enabled Axcient as a company to launch the next generation disaster recovery service.
Software Engineer
Commvault Systems
Oceanport, New Jersey
01.2012 - 12.2013
Working on Commvault's bare metal recovery project.
Backup/recovery of Iscsi disks, backup/recovery of Ubuntu based linux machines. Developed modules for restoring a physical machine to a Microsoft HyperV Virtual machine
Working on an NFS based data recovery mechanism using C++ which enables users to mount their backed up data as a NFS file system.
Software Engineer
Riverbed Technologies
Sunnyvale, California
05.2011 - 12.2011
Implemented a PcapDriver tool that takes two streams of ethernet packets as input and detects packet corruptions, reorders, duplicates etc
Worked on an effort to move one of the components of Riverbed's WAN optimization product Steelhead from TCMalloc to Jemalloc
Software Engineer
Symphony Services Private Limited
Pune, India
03.2007 - 09.2008
Developed a container based file system using C++ that is a thin provisioning mechanism for an embedded Windows kernel which helped extension and shrinking of the file system dynamically
Developed a file system volume recovery mechanism using C++ for the volumes that have on-disk metadata corruption
Education
Master of Science - Computer Science
Rutgers University
New Jersey
01-2011
Bachelor of Engineering - Information Technology
Vishwakarma Institute of Technology
Pune, India
01-2006
Skills
Observability
Distributed databases
Distributed file systems
Microservices
Kubernetes
Programming & Data structures (Java, Golang, C, C, Python, Scala)
Academic Projects
Modified the job scheduler of SLURM (https://slurm.schedmd.com) to generate job schedules that considers variations in power sources of nodes in the cluster.
Developed a mechanism to dynamically update linux kernel modules with their new versions on the fly.
Developed a basic networking service and website using Java that uses Apache HBase.
Affiliations
Outside of work, I enjoy keeping my brain occupied with cameras, Portfolio - https://nowhereinparticular.smugmug.com/