Summary
Overview
Work History
Education
Skills
Websites
Certification
Projects
Awards
Timeline
Generic
Sandeep Tipani

Sandeep Tipani

Dallas,USA

Summary

Results-driven data engineering professional and a Tech Leader with 12+ years in designing, developing, and optimizing data-driven applications handling 3PB/day volume of Walmart’s largest Omni Customer Interactions data to provide Real Time and Near Real Time Analytics to help business effectively launch campaign’s and usage of real-estate. Proven technical leader for 20 global associates, driving cross-functional collaboration and achieving impactful cloud cost optimizations that saved $6.5M annually. Proficient in Generative AI (LLMs, Prompt Engineering) and building prompt-driven solutions for complex retail and finance datasets. Committed to further disrupt the industry in a Technical Leadership Role.

Overview

12
12
years of professional experience
1
1
Certification

Work History

Staff Data Engineer |Technical Engineering Leader

Walmart Global Tech
01.2018 - Current
  • Designed and developed the Largest Omni customer Interactions datasets in Walmart which help business to make data driven decisions before launching any new features on Walmart E-commerce platform including usage of Real-estate
  • Led a global tech team of 20 associates and cross functional initiatives across the organizations to drive impactful results
  • Led organizational cloud cost optimizations by 18% resulting in 6.5M/year savings
  • Created the technical roadmaps and cloud cost budget’s for the domain and collaborated with engineers for efficient troubleshooting and coding to ensure prompt issue resolution and team effectiveness
  • Substantial experience in building resilient, reliable, durable data pipelines handling 3PB/day log, unstructured & structured data supporting Real Time and Near Real Time analytics using KAFKA, Spark Streaming, Kubernetes, GCP-Suite, BQ, Presto
  • Optimized the storage layer for interactions dataset which resulted in a savings of 500k/year
  • Implemented L3 layer table solutions using NVDIA GPU’s as a compute power which reduced the runtime by 65%
  • Collaborated with Platform teams to roll out the cloud cost optimizer tool to international domains
  • Extensive exposure to Gen AI – LLM’s, Prompt Engineering & Chat Bot’s to build prompt driven solutions on complex Omni retail and finance datasets resulting in saving 50% developer time and improving dynamic accessible patterns
  • Created a query mapper tool using the Gen AI - LLM’s to help the teams during the migration from old source to new gen source which saved 70% man hours
  • Played a key role in launching new features on Walmart E-Commerce platform by converting business specifications to technical structural specifications to allow data to flow seamlessly to all layers with zero to minimal changes across
  • Built executive dashboards to have a real time view of interactions data to understand different conversion metrics
  • Provided a real time dashboards to the Site Merchants to understand campaign performance and real-estate usage
  • Designed & Led the Traffic Sense UI work to replace Adobe Analytics tool and also to provide a detailed view marrying Paid and Un-Paid Marketing Vehicles to the financials across the key KPI’s like CTA, Sessions, A2C, Sales, ROI, AOU etc
  • Acted as a Data Quality Champion for the domain to ensure quality score >90% for all the assets
  • Implemented a framework to easy out the data quality implementation by providing a SQL interface which increased the Speed to Market time of a feature by 50% and also a very flexible way of controlling/adjusting the rules
  • Designed and implemented the changes to make the Walmart’s data assets MHMD compliant

Staff Data Engineer

Walmart Global Tech
05.2019 - 12.2020
  • Designed the data migration strategy from on premise to GCP for overall domain
  • Achieved 28% reduction in domain’s cloud expenditure by designing cost effective strategy to tune cloud clusters/jobs
  • Spearheaded the GCP cloud migration effort from On-Prem by uplifting the legacy pipelines using custom frameworks
  • Achieved 60% reduction in man hours with a custom framework built on Python/Spark-Sql to build/migrate data pipelines
  • Design and Developed a Data Validation framework to improve the data quality and reduce the anomalies
  • Uplifted the legacy data pipelines with new robust, scalable and reliable data pipeline using custom developed code
  • Designed and Developed a organization wide tool to monitor the infrastructure and also to provide cloud cost incurred at a job level with integrations to Pepper data and Spark lens to help developers optimize the spark jobs which reduced 60-70% downtime of jobs and significant impact in cloud cost reduction
  • Led and worked on a effort to set up the GCP clusters for overall domain enforcing the standards and isolating QA & Prod
  • Designed a cost-effective technical template for data teams utilizing Big Query and Druid on GCP as acceleration layers

Sr Data Engineer

Walmart Global Tech
01.2018 - 05.2019
  • Ingested streaming data from IBM MQ/Kafka using Spark streaming and loaded into Hive for consumption
  • Consumed data from Rest API in JSON/XML format and flattened it to store in Hive and applied business transformations
  • Participated in strategy to build a Data Lakes for the entire domain without replicating the efforts
  • Involved in setting up a Presto cluster for data organization doing the load and concurrency testing
  • Participated in code review sessions to ensure standards and data quality
  • Managed the security for overall domain datasets using Ranger and G-suits
  • Collaborated in launching new global products with platform teams to formalize the requirements and participate during the implementation/testing phases before rolling out to entire org

Sr Data Engineer

HCL America, INC
10.2016 - 01.2018
  • Involved in understanding the source data from various source systems and created mapping documents
  • Uplifted more than 80 existing data pipelines with new robust, scalable and reliable data pipelines built on Hadoop, spark
  • Automate the Deployment of new features using Ansible,Python,unix scripts and achieved 60-70% reduction in man hours
  • Developed Sqoop jobs for importing data from Netezza to HDFS
  • Developed data models to compliment Hadoop file system capabilities
  • Worked on a POC to implement SCD Type-1 and Type-2 in Hive
  • Orchestrated Control-M workflows invoking Spark and Sqoop jobs

Data Engineer

Tata Consultancy Services
06.2012 - 07.2015
  • Performed a thorough analysis of the AS-IS process and handled the mapping of nearly 50 tables that were used in the old system to the fields in new system understanding the business involved
  • Developed DataStage jobs to perform transformations using stages like Transformer, Aggregator, Merge, join, Lookup, Remove Duplicate, Funnel, Filter and created a flow using sequencers to load data warehouse
  • Experience in handling the Mainframe files using complex flat file stage and used other file stages
  • Developed COBOL modules to perform the sanity checks on the source files received through NDM, process the files and transmit them to downstream systems over SFTP

Education

Master of Science - Computer Science

The University of Central Missouri
12.2016

Bachelor of Technology - Information Technology

VNR VJIET
05.2012

Skills

  • Team Building
  • Strategic Planning
  • Technical Roadmaps
  • Leadership
  • Cloud Cost Budgeting
  • Product Management
  • Project Management
  • Cross Team Management
  • Engineering Excellence
  • Operational Excellence
  • Cloud Migrations
  • Tech Modernizations
  • Gen-AI
  • LLM’s
  • Prompt Engineering
  • Python
  • Scala
  • Unix Scripting
  • COBOL
  • JCL
  • Java
  • GCS
  • NVDIA GPU’s
  • Big Query
  • Big Table
  • IAM
  • Data Proc
  • Cloud Break
  • Serverless
  • Druid
  • Presto
  • Netezza
  • Spark
  • Hive
  • Spark-SQL
  • Sqoop
  • Kafka
  • Spark Streaming
  • Zookeeper
  • Google-IAM
  • Ranger
  • Kerberos
  • Airflow
  • Control-M
  • Automic
  • Oozie
  • Looker
  • Tableau
  • IBM Datastage
  • GITOps
  • CI/CD
  • Jenkins
  • Maven
  • Concord
  • Sonar Cube
  • Stack Driver
  • Splunk
  • Pepper Data
  • Sparklens
  • Agile
  • Scrum
  • Kanban
  • MHMD

Websites

Certification

  • Professional Google Cloud Architect, 03/23
  • IBM Certified Solution Developer – DataStage v8.0, 06/13

Projects

Twitter Sentiment Analysis, Designed Python scripts to use Twitter Streaming API for collecting real-time tweets. Recommendation System Based on Collaborative Filtering, Designed Scala and Spark jobs to analyze datasets and prepare training samples.

Awards

  • Excellence Award @ Walmart Global Tech, 11/20, Honored for creating reusable frameworks during cloud migration.
  • Impact Award @ Walmart Global Tech, 05/24, Received for leading organizational cloud cost optimizations.

Timeline

Staff Data Engineer

Walmart Global Tech
05.2019 - 12.2020

Staff Data Engineer |Technical Engineering Leader

Walmart Global Tech
01.2018 - Current

Sr Data Engineer

Walmart Global Tech
01.2018 - 05.2019

Sr Data Engineer

HCL America, INC
10.2016 - 01.2018

Data Engineer

Tata Consultancy Services
06.2012 - 07.2015

Bachelor of Technology - Information Technology

VNR VJIET

Master of Science - Computer Science

The University of Central Missouri

Similar Profiles

CREATE PROFILE
Sandeep Tipani