Hi, I’m
Anshu Kaushik
Irving,TX
Summary
Detail-oriented DevOps & MLOps consultant committed to improving data platforms system design and operations for reliable workflow management. Creates easy-to-follow guidelines and troubleshooting documentation for developers and Data Scientists. Meticulous Infra architect for Data Analytics and Machine Learning platforms for maintaining highly functional digital systems to provide uninterrupted workflows and consistent productivity. Oversees routine maintenance checks and software update schedules to reduce network downtime and likelihood of security compromises. Loves collaborating with customers and teammates to develop and implement creative technical solutions to challenging problems.
Big advocate of Open source solutions crafted and tailored to deliver using container based solutions on Azure kubernetes and Azure stack .
BigData & ML Infrastructure solutions architect with a strong background in administration and data solutions of more than 12 years.
Overview
12
years of professional experience
1
Certification
Work History
Teradata India Pvt. ltd, Airlines & Telecom
Cloud and BigData Architect
02.2019 - Current
Job overview
- Migration from Onprem Cloudera env to Azure stack (adls, databricks, event hub) ,data migration using Wandisco
- Migration of Onprem Rtudio/Jupyterhub env to Azureml & K8s based Machine learning platform
- Azure Subsription Management : Capacity Planning,responding to CVE's and azure advisory and managing multiple IAAS and PAAS solutions on Cloud (Azure)Terraform and Github actions for Azure resource provisoning
- Azure Kubernetes Administration ,Networking and Security
- Designing container based auto scalable solutions for data processing and exposing endpoints
- Install & manage apache Airflow on kubernetes for scheduling and orchestration across cloud and Onprem services
- Designed helm charts for nifi cluster installation on k8s providing end to end solution for automated cluster provisioning , CICD integration with nifi registry and github , full fledge security with https/tls and integration with azure active directory in fully automated way
- K8s & MLOps: plug and play Oauth framework using ouath2-proxy and Azure AAD/Github for open source tools like shiny ,bokeh, fastapi etc
- K8s & MLOps : CICD based on argos for k8s deployment and real time api's deployment for scoring
- AWX(upstream ansible tower) on k8s for automating daily Infra operational task like patching,security agent deployment,CVE's resolutions etc
- SRE based on nifi,prometheus,Grafana,loki,dynatrace and moogsoft
- Identified issues, analyzed information and provided solutions to problems
Teradata corporation ltd
Senior Administrator/Service Experience Manager
04.2016 - 10.2020
Job overview
- Airlines Customer , Telecom & banking, Hadoop/Cloudera Cluster Installation and Upgrade for cloudera and hadoop
- Expertise in administration of Cloudera & Hortonworks distributions of Hadoop, security, integration with third party tools, hands on experience with data migration tools like Cloudera BDR, NiFI
- Work with customers to design hadoop based technical architectures, migration approaches, and application optimisations that enable business objectives
- Capacity Management and Forecast of cluster resources ,Chargeback models for each business groups participating at cluster
- Operational roadmap, infrastructure logistics finalisation etc
- With customer
- NiFI Administration and Security and Integration Specialist
- SLA’s and compliance regulations
- Multi tenancy: Resource usage and isolation
- Log Monitoring automation and alerts response
HadoopAdministrator, Tata Consultancy Services
07.2013 - 03.2016
Job overview
- Hadoop Cluster installation and configuration
- Data governance and lineage using Navigator : Tag based policies and young data vs old data report for proactive optimisation of jobs
- Securing at role and file level using hdfs extended ACL’s
- Defining queues per team using fair scheduler
- Optimising Hive/Impala Queries
- Performance & Optimisation of Spark jobs
Open Source, Tata Consultancy Services
Database Administrator
03.2011 - 06.2013
Job overview
- System and Performance testing for databases before any deployment or configuration changes
- Configuring MySQL & Mongo DB servers for optimal use
- Regular monitoring of databases and server logs in production environment for checking system performance
- Administration of MySQL, Mongo DB databases and ETL layers :Symmetric DS
- DB designing and optimize table performance through indexing and explain plan
- Reviewing IT architecture plans for standards security practices in coordination with ISM:Information security management group
- CERTIFICATES
- Microsoft Certified : Azure Administrator Associate HDP Certified Administrator
- Teradata Database Certified Associate
Education
K.I.E.T ,UPTU
B.E from Electrical & Electronics Engineering
04.2010
Skills
- Cloud Architectures and Services
- Reliability systems using enterprise and open source tools like prometheus,grafana ,dynatrace ,moogsoft
- Solutions Architecture/Implementation
- Continuous Deployment
- Infrastructure Solutions
- Kubernetes Container ,K8s Operator ,helm
- NiFi on k8s,shiny on k8s ,streamlit on k8s
- Azure Cloud Security
- BigData platform using Cloudera and Databricks
- Multi-Factor Authentication & SSL/TLS
- Nifi
- MLOps using AzureMl and Kubeflow
- Awx tower on k8s
- Terraform for Azure provisioning automation
- Python(automation)
- Spark
- Airflow on K8s
- Kafka/eventhub
- Github Actions
- MySQL
- Apache Hadoop and Cloudera Solutions
- UNIX Shell Scripting
Accomplishments
- Alerting and monitoring solution for real time ingestion platforms for Streaming ingesting platform using nifi ,eventhub,prometheus for Enterprise Log Management
- Nifi on K8s deploy nifi cluster on kubernetes along with bells and whistle of observability and security
- Kubernetes Observability using prometheus , thanos , loki and tempo
Certification
- Certified Kubernetes Administrator : CKA
- Microsoft Certified: Azure Administrator Associate
- HDP Certified Administrator
- Teradata Database Certified Associate
Timeline
Cloud and BigData Architect
Teradata India Pvt. ltd, Airlines & Telecom
02.2019 - Current
Senior Administrator/Service Experience Manager
Teradata corporation ltd
04.2016 - 10.2020
HadoopAdministrator, Tata Consultancy Services
07.2013 - 03.2016
Database Administrator
Open Source, Tata Consultancy Services
03.2011 - 06.2013
K.I.E.T ,UPTU
B.E from Electrical & Electronics Engineering