DevOps Engineer heavily involved in the data science and related fields with both kubernetes and non kubernetes based platforms.
Working for Amtrak and then Nike
Amtrak
- Manage and Develop workloads for Quintiq on AWS environments
- Manage Load Balancers on the Frontend and Backend
- Maintain EC2 instances which served as servers for both front and backends
- Expertise in AWS including and not limited to EC2 instances, VPC, Cloudwatch, Cloudtrail, EMRs
- Shell and python scripting
- Deploy and maintain Jenkins pipelines for NodeJS applications using NPM
- Manage HTTP server for application hosting
- Host Jupyter notebook servers for data scientists
- Create and manage virtual environments for data scientists
- Dockerise the nodejs application and use compose for testing
- Manage Jenkins and use it for deployment
Nike
- Create a tool Pterradactyl which is now a open source, to manage AWS infrastructure using Terraform at its core, this enabled scalability with minimal management
- Work with NGAP team for maintaining EMRs, Airflows, DSTools environment for data science
- Use EMR for data science and support data science teams on using Jupyterhub on EMR
- Use Spark for data processing
- Manage AWS infrastructure using Terraform for the Lab222 team
- Work with AWS tools on EC2 instances, VPC, Cloudwatch, Cloudtrail, EMRs, EKS, ECR, Transit gateways, VPC, VPC endpoint, VPC endpoint services, Spark, Hadoop, Jenkins, ELBs and many more
- Design and implement EKS (kubernetes) based infrastructure to host and serve APIs generated from Data Science processes
- Design networking features needed for the NGAP team
- Experience with technologies like Spark, Kubernetes, Docker, Jenkins, Hive, Terraform, Helm
- Design and implement kubernetes controllers for the HyperFlow team(DevOps and Runtime under EAP) for cross account automation
- Implement HiveMetaStore access from Kubeflow
- Automate deployment of EKS and necessary components such as VPC, IAM, RBACs using terraform and helm
- Deploy and maintain about 40 EKS clusters using self designed automation processes
- Deploy and maintain ALB and nginx ingress controllers
- Deploy and maintain newrelic deployments
- Create and implement custom kubernetes controller for IAM automation in Kubeflow
- Manage VPC endpoints for the fulfillment application in the US EAST region and switch
to transit gateway
- Monitor applications using NewRelic
- Create Jenkins environment for CI/CD and develop custom Libraries for the same
- Create and Manage docker images using ECR and use EKS for containerization of application
- Manage EMR clusters with submitting jobs using spark, and help develop spark in Kubernetes for containerization of spark jobs
- Develop custom Kubernetes controller in a docker container automating creation of service accounts with namespace, using metacontrollers
CMS (through CVP Corp)
- Manage CI/CD Pipelines using Jenkins
- Manage Database clusters on AWS
- AWS to manage EC2, IAM, VPCs, SGs, EKS clusters
- Create and deploy Kubernetes Clusters on AWS and applications using Helm
- Use Terraform to manage the IAC for AWS stacks
- Deploy and manage EMR clusters for big data processing