Sivaramavenkata Botcha
Farmington Hills,MI
Summary
Experienced Big Data Professional with around 9 years of Administration experience working in different industries such as Healthcare, Retail and Finance. Professional working on Bigdata/hadoop, Cloud Infrstructure technologies, Linux administration and Devops. Hands on experience in setting up CDH, HDP and EMR infrastructure on prem and cloud(AWS and AZure cloud), managing and Leading operational activities in production level environments.
Overview
9
9
years of professional experience
Work History
Lead Bigdata Platform Administrator
Ally bank
09.2023 - Current
- Working as a hadoop SME for managing, maintaining, and ensuring the reliability of large-scale Hadoop infrastructure in a production environment.
- Experience in building and managing production level 200 node clusters using CDH 7.2.6. Also worked on the minor upgrades and major upgrades of hadoop using Cloudera manager.
- Expertise in sql and no sql required to manage the clusters using MYsql, postgresql .
- Expertise in assisting dev teams on the best practices on coding when it comes to hive, spark, and batch scheduling jobs.
- Support various bigdata platforms that we have both on prem and cloud infrastructure, such as Cloudera, Hortonworks and also latest EMR.
- Managed 3 clusters with around 200 node running on AWS cloud. Experience in managing using cloudera and also managing the cloud infrastructure and respective components such as database, linux ec2 and networking components.
- Worked in both the build team and operational activities. Giving ample of experience dealing with end users, trying to understand the dev team requirements and providing the infrastructure they need and managing the operational activities of the infrastructure
- Current stats of the datalake is a 100 node prod cluster managed using cloudera. Also mostly used toolsets are spark, hive, kafka, tez and respective cdh componets..
- Implemented security measures and best practices to safeguard sensitive Bigdata information and comply with regulatory requirements
- Designed and implemented disaster recovery and backup strategies to ensure high availability and data integrity of Bigdata systems.
Sr Bigdata Administrator
Quicken Loans
12.2019 - Current
- Work as a primary Bigdata Administrator, part of the Bigdata Infrastructure setup and support team. Technical work on a daily basis.
- Expertise in working on Authentication using Kerberos, Authorization of hadoop, currently using Ranger for this but experience working with sentry also and understanding of Encryption of data at rest and in transit both on prem and cloud infrastructure.
- Worked on Cloudera and Hortonworks distribution of hadoop, have 3 different clusters with 120+ nodes running using ambari 2.6.2,2.7.0 also managed in parallel CDH distribution of hadoop.
- Experience with Hadoop administration, various components of Hadoop including Spark, MapReduce, Tez, Hive, Pig, and several bigdata ecosystem components.
- Good Experience with Linux administration, working on the storage, networking using the command line and monitoring metrics on linux using tools like graphana and splunk.
- Experience in automating things on top of the clusters using ansible scripts and immense experience using automation scripts such as cloudformation and terraform to build up scripts for transient cluster creation.
- Experience on working with Sql datastores such as Mysql, auroraDB, postgresql to configure as backend databases for bigdata infrastructure setup.
- Experience working with NoSql databases such as Hbase, DynamoDb, MongoDB on both cloud and on prem.
- Experience working on Git , to store the terraform modules for automating the deployments of the current infrastructure.
- Experience working with ECS and EKS clusters on cloud to facilitate infrastructure setup for EMR and other bigdata distributions.
- Experience working on various cloud platforms such as Elastic Mapreduce, Azure Datalake. Was part of a project to migrate the Datalake from on prem infrastructure to cloud infrastructure.
Sr Bigdata Administrator
IQVIA
01.2019 - 12.2019
- Worked on the project of CDR to setup, Install, upgrading and maintaining of various Hadoop ecosystems with significant node count using Cloudera Distribution of Hadoop.
- Have Setup complete security around the CDR platform for all Authentication using Kerberos, Authorization using Sentry, Encryption at rest using KMS/KTS and Encryption in transit using TLS/SSL.
- Installed the cluster using the latest version of CDH 5.15 and later updated to version 5.16.
- Experience in management of Resource allocation for several services of Hadoop using YARN resource management. Also experience dealing with spark memory allocation without negotiation of resources for other services
- Have setup complete HA for all the components in Hadoop stack and had written backup scripts to takecare of data backups on a regular basis.
- Setup external connectivity to CDR clusters using ODBC and JDBC connectivity tools such as trifacta, mulesoft and using webhdfs API.
Hadoop Administrator
Shire Pharmaceuticals, Exton, PA
06.2016 - 12.2018
- Worked on the project of Data market Place to setup, Install, upgrading and maintaining of various Hadoop ecosystems with significant node count of 150+ nodes.
- Installed the cluster using Cloudera Distribution of Hadoop (CDH- 5.7) and later updated the version to 5.8 to 5.13.
- Installed the cluster using Hortonworks Distribution Platform (HDP- 2.4.X) and later updated the version to 2.5 and currently using the latest version of HDP 2.6. Experience in Installing several clusters using Hortonworks distribution with ADLS and Blob as the underneath storages on Azure Cloud platform.
- Strong experience in working on AWS and related services such as aws management, lambda, ec2, Terraform, S3 management related to Hadoop distribition.
- Worked on the project of Data market Place to setup, Install, upgrading and maintaining of various Hadoop ecosystems with significant node count of 150+ nodes.
- Installed the cluster using Cloudera Distribution of Hadoop (CDH- 5.7) and later updated the version to 5.8 to 5.13.
- Installed the cluster using Hortonworks Distribution Platform (HDP- 2.4.X) and later updated the version to 2.5 and currently using the latest version of HDP 2.6. Experience in Installing several clusters using Hortonworks distribution with ADLS and Blob as the underneath storages on Azure Cloud platform.
- Experience in working on Informatica ICS and BDM to ingest data from Traditional RDBMS sources to Hive and HDFS.
- Experience with various flavors and versions of Linux, also setting of services like NTP, Chrony. Experience in upgrading, patch, troubleshooting, performance tuning, backup, recovery and cloning of Oracle Database in Red Hat Enterprise Linux (RHEL).
Education
Master of Science - Computer Science
University of Missouri
Kansas City12-2015
Bachelor Of Technology - Electronics And Communication Engg
Gayatri Vidya Parishad College of Engg
Visakhapatnam,India06-2014
Skills
- Hadoop Administation
- Linux Administration
- Cloud Infrastructure - AWS and Azure Cloud
- Datalake Setup and Operational Activities
Timeline
Lead Bigdata Platform Administrator
Ally bank
09.2023 - Current
Sr Bigdata Administrator
Quicken Loans
12.2019 - Current
Sr Bigdata Administrator
IQVIA
01.2019 - 12.2019
Hadoop Administrator
Shire Pharmaceuticals, Exton, PA
06.2016 - 12.2018
Master of Science - Computer Science
University of Missouri
Bachelor Of Technology - Electronics And Communication Engg
Gayatri Vidya Parishad College of Engg
Similar Profiles
Saravanan PonnusamySaravanan Ponnusamy
Principal Software Engineer at Ally BankPrincipal Software Engineer at Ally Bank
BROOKE FORNGESBROOKE FORNGES
Data Security Engineer Intern at Ally BankData Security Engineer Intern at Ally Bank
Jennifer AndersonJennifer Anderson
Operational Risk Manager at Ally BankOperational Risk Manager at Ally Bank