Anil Bethapudi
Portland,OR
Summary
- 18 Years of experience with ERP Application Development, Technical Consulting, and implementation of CC&B (customer management and billing) solutions, Supply Chain management and HRMS in Oracle EBS 11i and R12 and S4 Hana.
- Over 10 years of Valuable experience in Information Technology Architect, Application Design, Development, Supporting Oracle SOA and MuleSoft 4.2 under Verity of Environments.
- Having 7+ years of hands-on Experience on Hadoop ecosystem components like Azure Data bricks, Azure Data Factory including their installation and configuration.
- Involved in design discussions and implemented performance and cost-efficient solutions.
- Interlock with business users and analysts to understand the requirements and deliver solutions for them.
- Established a framework for all the pipelines and coding standards.
- Involved with numerous POCs to evaluate different technologies to handle business challenges.
- Worked with different teams and lead them throughout the life cycle of the solution implementation.
- Demonstrated the design in show & tell sessions to the stakeholders and top-level management.
- Extensively worked on building agents/bots using databricks agentbricks.
- Established connection from databricks to non-cloud databases using lake federation and delta share. Built numerous AI/BI dashboards with embedded Genie.
- Involved data extraction from ERP systems like S4 Hana to databricks using SAP Datasphere and BDC
- Good Knowledge in creating event processing data pipelines using Kafka and Spark Streaming.
- Designing applications to ingest the real-time network sensor event data using AWS cloud services. Implementing real-time event data ingestion using Spark, SQS, Kinesis, S3, SNS and HBase on EMR.
- Performance tuning and monitoring the jobs using Grafana, Cloud Watch and Spark UI.
- Implemented CICD with azure Dev-ops and Databricks Asset Bundles.
- Hands on experience in AWS Cloud in various AWS services such as Redshift cluster, Route S3 domain configuration Migrated an existing on-premises application to AWS.
- Used AWS services like EC2 and S3 for small data sets. Performed presales activities like doing proof of concept, API estimates and API monetization.
- Expert in writing SQL queries, Cursors, Sub programs like Procedures, Functions, Views, Indexes and Packages & Triggers in PL/SQL blocks and Performance Tuning using Explain Plan.
Overview
19
19
years of professional experience
Work History
Principal Systems Data Architect
Qorvo
09.2024 - Current
- Design and build Databricks pipelines
- Establish a robust coding framework
- Implemented CICD with Jenkins and Databricks Asset Bundles.
- Implemented reusable python assets and frameworks.
- Developed AI agents using databricks Agentbricks.
- Mane deliverables with different business stakeholders
- Integrate S4 Hana systems with Databricks.
- Designed scalable data architectures to support business intelligence initiatives.
- Developed and implemented data integration solutions using ETL tools.
Lead Data Engineer
Nike
06.2023 - 08.2024
- Migration of the pipelines from AWS to Databricks
- Develop the pipelines integrated with technologies like snowflake, Kafka.
- Implemented CICD with Jenkins and Databricks Asset Bundles.
- Implemented reusable python assets and frameworks.
- Implemented data extraction and transfer solution using AWS services like EC2, s3 and Kinesis.
- Launched Airflow instances using Cloud formation templates.
- Established continuous query testing of the Lambda and Airflow code through CICD pipeline.
- Build integrations using Mulesoft.
- Designed and implemented scalable data pipelines to enhance data accessibility across departments.
- Led cross-functional teams in the development of data integration strategies, optimizing workflows and processes.
- Mentored junior engineers on best practices for ETL processes and data modeling techniques.
- Spearheaded initiatives to improve data quality, ensuring accuracy and consistency in reporting metrics.
Lead Big Data Engineer
Hewlett Packard Enterprise
03.2022 - 02.2023
- Automated the extraction of data from Snowflake and dispatching it to Snowflake using Airflow DAGs
- Created Continuous Integration and Continuous Deployment pipeline using GitHub, Jenkins, and Circle CI
- Implemented data extraction and transfer solution using AWS services like EC2, s3 and Kinesis.
- Launched Airflow instances using Cloud formation templates.
- Established continuous query testing of the Lambda and Airflow code through CICD pipeline.
- Established data transfer logic between Airflow and Lambda to exchange JSON payloads.
- Experienced with IDE tools like PyCharm, VS Code and Git integration.
- Hands on experience in Snowflake Cloud data warehouse in creating objects, stages, and file formats.
- Experience in developing and designing POCs using Python and deployed on the Yarn cluster, compared the performance of Spark, with Hive and SQL/Teradata.
- Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs and Python.
- Performed advanced procedures like text analytics and processing, using the in-memory computing capabilities of Spark using Scala.
- Importing the data into Spark from Kafka Consumer group using Spark Streaming APIs.
- Worked in tuning Hive and Pig scripts to improve performance.
- Knowledge on handling Hive queries using Spark SQL that integrate Spark environment.
- Implemented CICD with azure devops.
- Configured build scripts for multi module projects with Maven and Jenkins CI.
Lead Integration Engineer
KinderCare Educations LLC
02.2018 - 02.2022
- Developed simple and complex MapReduce programs in Java for Data Analysis on different data formats.
- Developed MapReduce programs that filter bad and un-necessary records and find out unique records based on different criteria.
- Developed Secondary sorting implementation to get sorted values at reduce side to improve MapReduce performance.
- Implemented Custom writable, Input Format, Record Reader, Output Format, and Record Writer for MapReduce computations to handle custom business requirements.
- Implemented MapReduce programs to classify data records into different classifications based on different type of records.
- Experience with creating ETL jobs to load JSON data and server data into MongoDB and transformed MongoDB into the Data Warehouse.
- Experience in developing and designing POCs using Scala and deployed on the Yarn cluster, compared the performance of Spark, with Hive and SQL/Teradata.
- Created Ab Initio graphs that transfer data from various sources like Oracle, flat files and CSV files to the Teradata database and flat files.
- Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for hive performance enhancement and storage improvement.
- Performed advanced procedures like text analytics and processing, using the in-memory computing capabilities of Spark using Scala.
- Responsible for performing extensive data summarization using Hive.
- Importing the data into Spark from Kafka Consumer group using Spark Streaming APIs.
- Developed Pig UDF's to pre-process the data for analysis using Java or Python.
Lead Engineer
Cummins Inc
02.2016 - 01.2018
- Designed applications by working closely with Enterprise architect and business, to understand requirements and problem statements, proposed the technical solutions.
- Designed and developed robust generic data pipeline framework to process multiple format vendor files (Fixed, Delimited, JSON) in real-time and batch mode using Spark-Kafka streaming and Spark batch.
- Profile, filter and transform the data per business requirements to store to the Enterprise Data Warehouse (Hive/ HDFS), generated the enterprise format outbound file (JSON) to transfer to downstream applications.
- Designed and implemented Data Lake by importing data from multiple RDBMS to the Hadoop platform using Sqoop.
- Implemented robust generic data pipeline framework to process multiple format vendor files (delimited, XML and SAS) using Spark by applying the business transformation to understand and assess the risk.
- Stored the transformed data in Enterprise Data Warehouse (Hive), generated delimited outbound file to securely transfer downstream.
- Led the production deployment and performance tuning of the jobs by tweaking memory of distributed systems.
- Led the development team and helped to prioritize the deliverables by removing the roadblocks.
BIG Data Sr Engineer
McDonalds
09.2012 - 01.2016
- Coordinated with business customers to gather business requirements, also interacted with other technical peers to derive technical requirements and delivered the BRD and TDD documents.
- Worked in 250 Node Hadoop cluster and building and operating large-scale, distributed enterprise grade Big Data applications using Hive, MapReduce, Hbase, PIG, Yarn, HDFS.
- Optimized the data warehousing environment with Informatica and Hortonworks.
- Building Real Time Processing Topologies of Sprint subscriber Behavior Data using Kafka, Storm Spouts, Bolts and Trident topologies.
- Loading streaming data using kafka and processing Using Storm.
- Implemented Spark using Scala and SparkSQL for faster testing and processing of data and Developed HbaseBolts in Storm topologies.
Sr Integration Engineer
GSK (GlaxoSmithKline)
02.2012 - 08.2012
- Responsible for attending scrum meetings and discuss about deliverables.
- Good working knowledge on installing and configuring the Hadoop Ecosystem Components.
- Responsible for analyzing data from external sources and deciding on the Storage Structure in HDFS.
- Move the Financial and Non-Financial data (in flat files) generated from various sources to HDFS path.
- Develop Pig Scripts for De - Normalizing / structuring the raw data from HDFS path.
- Set up Sqoop Jobs to export the pig processed data into Relational Database; and HiveQL to load the data into Hive tables.
- Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
- Support/Troubleshoot MapReduce programs running on the cluster.
- Create external tables, load data, and write queries in Hive.
- Develop scripts to automate routine DBA tasks using Linux/UNIX Shell Scripts/Python.
Integration Sr Consultant
Indovision
10.2010 - 01.2012
- Design the RTF environment on AWS using Docker and Kubernetes.
- Create High level Design for the integration of the various systems using MuleSoft Anypoint platform.
- Integrated Mule with Grafana for log dashboards.
- Worked on sizing the pods in K8s cluster.
- Setup the coding standards, naming conventions
- Participate in Code reviews, resource hiring, knowledge sharing and ensure quality of the work.
- Setup the Continues Integration and Continues Deployment (CI/CD) tools using Github actions and Git.
- Developed the Out of the box custom process to integrate to the existing legacy applications.
- Implemented the business functionally by developing the Mule flows using connectors like File, FTP, and JMS.
- Worked on integrations with different message brokers like JMS and Solace MQ.
- Supported the UAT with the business.
SQL Sr Developer
Daarasat
11.2008 - 09.2010
- Developed processes using technology adapters (FTP Adapter, DB Adapter) of Oracle BPEL and tune them for the expected load, Transformations, Exception Handling, and use of Third Party JDBC Drivers.
- Configured Oracle Applications adapter for invoking PL/SQL API to send data to EBS, and for receiving Order status from EBS.
- Worked with Oracle Fusion Middleware Adapters, Oracle JDeveloper 10.1.3.4, Java, JSP, HTML and JavaScript.
- Configuration and Maintenance of JDBC, JMS, JNDI and other J2EE Services with WebLogic 10.3.
- Work with other programmers/designers/architects to make sure that the components meet integration requirements.
- Developed Technical Specification for on boarding three new EDI Customers (B2B).
- Create the technical design document, MD70 as part of the AIM. MD70 will include the process flow and the technical features used to develop the component, input and output specification and setups needed.
- Involved in development of interfaces which will extract the data from base tables into flat files using UTL_FILE.
- Registered PL/SQL packages, Forms, Menus and reports as concurrent programs.
- Developed PO Open Interface to upload the Standard Purchase orders data into Oracle PO Base tables by using PL/SQL Program.
- Excellent documentation with respect to solution design, User Acceptance, test scripts, Production setup document, and issue logs and document standardization in line with AIM Methodology.
- Prepared technical documents (MD070) for reports, interfaces, and concurrent programs as per client business logic.
- Define value sets, concurrent programs, request sets, registering reports and PL/SQL Procedures and Packages.
Sr SQL Developer
Boom TV
02.2007 - 10.2008
- Custom validations in Oracle using cursors, triggers, procedures, and functions which only allow specific business functions suggested by the client thereby restricting the improper flow for a better business approach as Techno functional consultant.
- Developing reports using Business Objects – Crystal Reports for financial analysis, reconciliation, etc...
- Data corrections – problem finding using custom scripts written in SQL/PLSQL and problem-solving using SQL/PLSQL with cursors, triggers, procedures, and functions.
- Client business process analysis – Understanding client business processes, suggesting / improvising the business scenarios, etc... and implementation of few modules as per client’s requirement.
- Regular maintenance of billing activities and daily issues resolving using SQL/PLSQL.
- Data porting of all the subscriber base into MQS using SQLLOADER and inserting the data accurately into various table by writing stored procedures, triggers, and cursors.
- Developed a custom workflow for Customer Care module at DB level where there is no feasibility for the client required workflow within the product. Good level of using triggers, stored procedures, and functions for this workflow setup.
SQL Developer
Star India Ltd.
07.2006 - 01.2007
- Coordinating with team members and analyzing the client's business process for requirement mapping, system design integration and product implementation.
- Accurate configuration of business processes as per the scope defined and workflows identified within the agreed time lines (via Business Process Document and project plan).
- Write / enhance the procedures / triggers and other relevant objects of the DB using Oracle PL/SQL for meeting the analysis requirements as per scope defined.
- Worked on performance tuning both script level and server level.
- Preparation of Acceptance Test Cases. Facilitate to conduct the acceptance tests during implementation conforming to the defined scope with prior acceptance of the solution by the client.
- Developing simple / complex custom reports confirming to clients processes using Crystal Reports.
- Limited to the identified business processes and workflow, train or hand hold the team of users at customer's location. Effectively imparting training to the users or user groups at customer's location. Training PPTs and relevant documentation and training environment like the solution configured as per the client's processes along with sample data.
- Resolving queries/issues at production level and extending onsite support including maintenance support.
Education
Master of Science - Computer Applications
Acharya Nagarjuna University
India06-2006
Skills
- Hadoop Core Services: HDFS, MapReduce, Spark, Yarn
- Hadoop Distribution: Hortonworks, Cloudera
- NoSQL Databases: MongoDB
- Hadoop Data Services: Hive, Pig, Impala, Sqoop, Flume, NiFi, Kafka, Storm
- Middleware: MuleSoft, Oracle SOA
- Programming Languages: Core Java, Scala, Python
- Databases: Oracle, MySQL, SQL Server
- ETL: Alteryx
- Operating Systems: UNIX, Windows, LINUX
- Development Tools: Eclipse, NetBeans, IntelliJ, Anypoint Studio
Timeline
Principal Systems Data Architect
Qorvo
09.2024 - Current
Lead Data Engineer
Nike
06.2023 - 08.2024
Lead Big Data Engineer
Hewlett Packard Enterprise
03.2022 - 02.2023
Lead Integration Engineer
KinderCare Educations LLC
02.2018 - 02.2022
Lead Engineer
Cummins Inc
02.2016 - 01.2018
BIG Data Sr Engineer
McDonalds
09.2012 - 01.2016
Sr Integration Engineer
GSK (GlaxoSmithKline)
02.2012 - 08.2012
Integration Sr Consultant
Indovision
10.2010 - 01.2012
SQL Sr Developer
Daarasat
11.2008 - 09.2010
Sr SQL Developer
Boom TV
02.2007 - 10.2008
SQL Developer
Star India Ltd.
07.2006 - 01.2007
Master of Science - Computer Applications
Acharya Nagarjuna University