LAVAN KUMAR GALLAVARI
Tampa,FL
Summary
I bring over 12+ years of experience as a Senior Data Engineer specializing in Big Data engineering and cloud solutions. I have expertise in developing and optimizing ETL processes using Spark and leveraging various AWS services and with a background in production deployment and support. Moreover, I have coordinated with multiple teams and handled the scrum board for Jira stories, ensuring effective collaboration and task management.
I am well-versed working with team(s) to address intricate data challenges and pioneering innovative data pipeline architecture approaches. Additionally, my certification as an associate developer for Apache Spark 3.0 attests to my expertise in this area. I have extensive experience in writing and optimizing Spark SQL, leveraging it to efficiently query and process large-scale datasets, contributing to efficient data processing and analysis.
Furthermore, I have experience and expertise in architecting, configuring, scheduling, and monitoring pipelines using Apache Airflow. This includes developing intricate Directed Acyclic Graphs (DAGs) that orchestrate and automate workflow processes, ensuring efficient and reliable data processing and delivery.
Overview
14
14
years of professional experience
Work History
Senior Data Engineer/ Onsite Coordinator
Tata Consultancy services
Oldsmar
08.2021 - Current
- Gathering requirements from clients and analyzing them
- Preparing design documents and flow diagrams to understand the requirements
- Analyzing the feasibility of the requirements in the existing system
- Once a change is deemed fit for construction, the total cost in terms of effort and schedule is calculated,this takes into consideration the nature of change, number of impacted components and the skills and availability of resources
- Attending customer/operational users queries and providing a solution
- Performing unit testing for the completed work items
- Review code changes done by team members to ensure they are fulfilling the user requirements
- Check the code for correctness and provide required inputs to update the same if in case something is missing using Assent
- Deploying the QA approved and wellness certified product into production to be used by actual users
- Provided global thought leadership in analytics solutions to benefit customers
- Extract the raw data and process it to the needs of customer
- Adept in troubleshooting and identifying current issues and providing effective solutions
- Managed performance monitoring and tuning while identifying and repairing issues within the database realm
- Assisted solution providers with definition and implementation of technical and business strategies
- Collected information by meeting with database architects and administrators to determine client system limitations for capacity and load for data processing projects
- Collaborated with solution architects to define database and analytics engagement strategies for operational territories and key accounts
- Promoted customer success in building and migrating applications, software and services on AWS platform
- As part of performing above responsibilities, worked on the following technologies/methodologies: Loading the data from the different Data sources like (Redshift, Postgres, Hive) into HDFS using SparkQL and processing the raw unstructured data
- Used Scala for fast and general processing engine compatible with Hadoop File System
- Setting up right AWS EMR Cluster, which includes configuring different components of HADOOP
- Analyzed large Hive data sets by designing optimized queries
- Used advanced window SQL functions to quickly process the data with less resources
- Ensured data accuracy through regular testing and validation procedures prior to deployment in production environments.
- Setting up SNS and SQS for Application integration with other AWS services (S3 and Lambda)
- Configuring Auto Scaling groups, Cloud watch alarms to ensure high scalability of applications that are critical to business
- Maintaining right IAM roles and security groups for smooth flow of applications
- Configuring S3 archival components that are cost effective and maintaining the S3 versioning in the areas needed
- Effectively setup and Maintain GITLAB as code Repository
Technical Lead at Offshore
Tata Consultancy services
02.2015 - 08.2021
- Gather requirements and perform system, impact analysis by going through the functional documents, business requirement documents and understanding the value it is going to add for the client
- Compute the dependency between various systems /applications and propose the possible solutions (Application architecture) to design the application
- Provide training to the team members on business process flow and technical topics for better understanding of module
- Design and Develop industry-specific solutions based on client needs and manage and upgrade existing applications
- Carry out detailed analysis to understand requirements and create code and/or build solutions as per requirements in development/maintenance projects in accordance with coding standards
- Decide and specify the final best solution which maximize the benefits to the client in terms of cost/quality/performance
- Perform unit testing per test plans and test cases
- Coordinate with team and ensure necessary documentation (Run-book, Design architecture, Application Maintenance Manual & Infrastructure details) is made
- Review the documents thoroughly and hand it over to the client and Operations for maintaining the application effectively
- Ensure latest code is updated in code repository
- As part of performing above responsibilities, worked on the following technologies/methodologies: Migrated the applications that are orchestrated using the Tibco framework to Apache Airflow
- Configured multiple Airflow Tasks and DAG's using the centralized utilities in Airflow
- Removed the on prem dependencies by using cloud resources such as EC2 and S3 and used Scala as programing language to process the data
- Played a key role in DB migrations (Netezza to Redshift, Sybase to Postgres)
- Performed repository migrations from SVN to GITLAB
Module Lead/Technical Associate
Tata Consultancy services
Chennai
07.2011 - 02.2015
- Implemented quality control objectives to resolve production issues, optimize product reliability and reduce costs
- Troubleshoot and resolve production issues to execute smooth, streamlined programs
- Resolves application related issues experienced by end-users
- Managed junior developers by delivering consistent coaching and constructive feedback
- Identify and assign roles to team members and set project goals
- Provide training to the team members on business process flow and technical topics for better understanding of module
- Produced daily operations report, forecasting staffing needs and production volume to support cost-reduction and service improvement
- Supervised production schedules, production quality and on-time delivery
- Mitigated production issue risks by identifying potential problems and taking effective action
Java developer/Trainee associate
Aequor Information Technologies
Gurgaon
07.2010 - 07.2011
- Learnt new technologies and company principles(Java,SQL, C#)
- Participate in formal training process, be it face to face sessions or online activities
- Complete tasks, assessments, assignments that are set by the Mentor
- Abide by company rules and principles
Education
BACHELOR OF TECHNOLOGY -
Skills
- Spark sql
- Scala
- Airflow
- Python
- Java
- Oracle
- Postgres
- Redshift
- Hive
- Sybase
- Netezza
- AWS cloud Services
- GITLAB
- Maven
- Gradle
- Monitoring
- AWS Cloud watch
- Unix shell script
- IntelliJ
- Eclipse
Timeline
Senior Data Engineer/ Onsite Coordinator
Tata Consultancy services
08.2021 - Current
Technical Lead at Offshore
Tata Consultancy services
02.2015 - 08.2021
Module Lead/Technical Associate
Tata Consultancy services
07.2011 - 02.2015
Java developer/Trainee associate
Aequor Information Technologies
07.2010 - 07.2011
BACHELOR OF TECHNOLOGY -
Similar Profiles
Vrunda ChindhadeVrunda Chindhade
HR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services LimitedHR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services Limited
Sreya BasuSreya Basu
Procare Analyst at Tata Steel(Tata Consultancy Services)Procare Analyst at Tata Steel(Tata Consultancy Services)
Arun VArun V
Program Manager at Tata consultancy services | Financial Ombudsman ServicesProgram Manager at Tata consultancy services | Financial Ombudsman Services
Naimish AgnihotriNaimish Agnihotri
Assistant Delivery Manager at Tata Consultancy Services, Global Shared ServicesAssistant Delivery Manager at Tata Consultancy Services, Global Shared Services
Jeremy KellerJeremy Keller
Section Chief at US Air Force- Dept. of DefenseSection Chief at US Air Force- Dept. of Defense