Summary
Overview
Work History
Education
Skills
Accomplishments
Timeline
Generic
Anuj Shrotriya

Anuj Shrotriya

Alachua,FL

Summary

Accomplished senior engineering professional with extensive experience in data architecture, pipeline development, and big data technologies. Demonstrated success in streamlining data workflows, improving system efficiency, and spearheading business intelligence projects. Collaborative team player thriving in dynamic project environments, dedicated to achieving meaningful outcomes through innovation and teamwork. Proficient in SQL, Spark, and cloud platforms, known for strategic data management and adept problem-solving skills.

Overview

14
14
years of professional experience

Work History

Senior Data Engineer

Microsoft
05.2023 - Current
  • I am currently part of XBOX- Publishing Game Services team. I am working on designing and implementing privacy and security compliant Big Data Pipelines for Game telemetry data. Also, working on enabling data analytics for partners with data governance.

Senior Software Engineer

Microsoft
12.2020 - 05.2023
  • I was part of Supply Chain Engineering team which powers Data Analytics / Business Intelligence(BI) for devices org. My team owned the data platform, I worked on BigData pipeline processing using Apache Spark, Hadoop, HD Insight, DataBricks, SQL, ADL, ADF, ADX.
  • Leading the ETL processing framework
  • Setting strategic direction, design, and development of the next version of the ETL framework.
  • Continuous evangelization and setting guardrails of the framework across cross-functional teams.
  • Refactoring architecture from the legacy framework by developing tools for large-scale code migration.
  • Executed end-to-end lifecycle from design, deployment, validation, and operationalization which included - 150+ pipelines by demonstrating core capabilities of the framework like performance, DQs, Local testability, logging, data discovery, data lineage, a significant reduction in LOC etc.
  • Increase TCO and ROI
  • Shipped crucial features like inventory aging projection that resulted in $$ revenue savings.
  • Worked on Control tower initiative for Supply Chain which is expected to enable very key business scenarios again resulting in considerable $$ savings.

Senior Software Engineer

Microsoft
07.2019 - 12.2020
  • I was part of the Product 360 team, which provides Data Analytics & Business Intelligence (BI) across all Azure Services including usage across subscriptions mapped with ARR (Anual Recurring Revenue), adoption, and health check via CRM integration, etc. My team owned the data platform, I worked on BigData pipeline processing using Apache Spark, Hadoop, HD Insight, DataBricks, SQL, ADL, ADF, ADX.
  • Application Modernization - Took a vague problem for the app’s performance improvement and delivered results successfully. Leveraged Azure AppInsights Telemetry.
  • Increased user experience and adoption - by simplifying data analysis for customers by formalizing and providing new Kusto-based APIs.
  • New engineer onboarding & training, mentoring, recurring code review, Best practices, On-Call support, and continuous process improvements.

Senior Software Engineer

Microsoft
09.2015 - 07.2019
  • I was part of the Microsoft- Azure Intelligence Platform(AIP) team, which provides the critical function of data integration for Azure services. The function was incepted to drive the principle of quick problem-solving, incubation, and innovation to attain a leading Public Cloud provider position by bringing feature parity. My team was the first to re-platform Enterprise Data Lake (EDL) from On-Prem to Azure Cloud. I worked on the following tech stacks - Azure blob storage, BigData pipeline processing using Apache Spark, Hadoop, HD Insight, DataBricks, SQL, ADF, ADX.
  • Key Business impacts projects. Solved the problem of 'Customer Master', which many teams in Microsoft had been trying to solve. This gave a considerable boost to the field to drive Azure sales. Delivered a critical project 'Microsoft Cost Management' that required cross-group collaboration under strict timelines. As part of this project, Azure enterprise customers could see their costs refreshed 3 times a day as opposed to once a day. The successful execution brought parity with AWS. Redesigned the system which resulted in pipeline processing time reduction from 4-5 hours to 20 mins. With this, Azure Enterprise customers could now see their costs refreshed 6 times a day. This also resulted in a significant reduction in operating costs (DRI, Infrastructure).
  • Engineering Excellence:
  • Developed big data processing framework- Developed framework to reduce complexity and enable ease of use by abstracting crucial functionalities like load balancing, DQs, read/write standardizations, file size control, and unit testability from developers. This increased efficiency and reduced the development timeline.
  • Data Accuracy Solution: Worked with upstream and downstream partners and came up with a solution to act as a single source of truth to identify data discrepancies across the system. This helped attain strict SLAs and proactively alert and, identify issues before customers and stakeholders.
  • Business Continuity and Disaster Recovery (BCDR) Automation - Automated BCDR controls to reduce the overall switch time from 2 hrs. to less than 15 mins.
  • Infra Cost Optimisations: Drove the Infrastructure to spend on HDI spark to less than 20% by introducing cluster scale-up/down activities as part of ADF workflow.

Software Engineer

Microsoft
09.2014 - 09.2015
  • I was part of the Shared Data Platform team at Bing org.
  • Onboarded backend of Microsoft’s Central Revenue and sales reporting system (MSSales) to Shared Data Platform (SDP). SDP offers distributed Database solution (called Stripe). It offers advantages like Low operating cost, faster query responses, easy to scale. Project was successfully delivered under strict timelines.
  • Optimized Big data pipelines on Cosmos. Developed tools for the autogeneration of code that significantly reduced the development time.
  • Developed expertise on SDP solutions like Sangam (Orchestration service for cosmos pipelines), Stripe (Distributed Database solution), DVS (Data virtualization services).

Software Engineer

Microsoft
07.2011 - 09.2014
  • I joined Microsoft Bing - Ads org as early in my career Software engineer. I quickly excelled and attained engineering Excellence Champion.
  • Perf tuning and optimization of SQL Database and queries.
  • Migrate heavy processing flows from SQL to BigData platform- Cosmos.

Education

M.Tech - IT

ABV- Indian Institute Of Information Technology & Management
01.2011

Skills

  • Apache Spark
  • Azure Data Lake
  • Azure Data Factory
  • Azure Data Explorer
  • C#, Powershell, Scala
  • SQL
  • Microsoft SQL Server
  • Data Structures & Algorithms
  • OOPS
  • Big data processing
  • ETL development
  • Git version control
  • Data pipeline design
  • Data modeling
  • Spark development

Accomplishments

  • Got promoted each year for 3 consecutive years for outstanding contributions and attitude.
  • Got Recognition for 'Outstanding contributions' in Editorial All Hands (held quarterly).
  • Participant in Microsoft HiPo (high potential) program for FY17.
  • Qualified thrice for ACM-ICPC (international collegiate programming contest) regionals.
  • Qualified GATE twice, with AIR-648 in GATE-2009(CS & IT).

Timeline

Senior Data Engineer

Microsoft
05.2023 - Current

Senior Software Engineer

Microsoft
12.2020 - 05.2023

Senior Software Engineer

Microsoft
07.2019 - 12.2020

Senior Software Engineer

Microsoft
09.2015 - 07.2019

Software Engineer

Microsoft
09.2014 - 09.2015

Software Engineer

Microsoft
07.2011 - 09.2014

M.Tech - IT

ABV- Indian Institute Of Information Technology & Management

Similar Profiles

CREATE PROFILE