Balasiva Pindra

Architected Enterprise Data Warehousing Solutions:

• Lead an architecture of comprehensive data warehousing solutions for Rivian, employing cloud technologies including AWS, Databricks, and Bigdata (PySpark)

Developed Scalable Data Pipelines:

• Engineered and implemented scalable data pipelines to efficiently extract, transform, and load large volumes of both structured and unstructured data with in Streaming and batch
• Achieved a remarkable 30% increase in data processing speed, showcasing a commitment to optimizing performance and operational efficiency

Enterprise Data Warehouse Management:

• Established and managed enterprise data warehouse comprising over 1000 tables and 100 pipelines, orchestrated seamlessly through AWS Airflow
• Developed, implemented and maintained data analytics protocols, standards, and documentation

Custom Packages for Self-Service Applications:

• Designed and implemented custom packages to empower business users with self-service applications, enabling them to effectively utilize data pipelines within platforms like SharePoint and AWS S3
• Streamlined user accessibility and contributed to heightened organizational efficiency

Cross-Functional Collaboration:

• Collaborated effectively with cross-functional teams including commercial, supply chain, product development, manufacturing, and service teams.Translated their analytical needs into actionable insights, leading to the development of models and rigorous data quality checks for enhanced accuracy and relevance across diverse business functions

CI/CD Implementation:

• Designed and developed CI/CD processes using GitLab, Terraform, Pulumi, and CloudFormation for infrastructure as code

Migration of On-Premises Data Infrastructure:

• Led the migration of on-premises data infrastructure for manufacturing data
• Which results in a 25% reduction in infrastructure costs while simultaneously improving system availability and scalability.

AWS Server Configuration and Routing:

• Configured ALE Server on EC2 instances within a VPC, employing ALB, NLB, ELB, and router configurations, seamlessly integrated with Route 53

CI/CD Deployment with Jenkins:

• Implemented AWS infra deployment using Jenkins CI/CD, featuring automatic test report generation using PyUnit & PyTest.

Containerization for Critical Jobs:

• Containerized critical jobs exceeding Lambda runtime, deploying them in AWS ECS
• Executed Data Quality checks in Python & Spark on Snowflake data, with parallel processing for multiple queries and aggregate data verification against defined standards

Data Quality Visualization and Reporting:

• Designed data quality jobs that visualize various data plots in a graphical format, sending daily email reports to responsible stakeholders

Real-Time Streaming with Kafka:

• Developed Java producers for Kafka to subscribe to ZeroMQ streaming data in real-time
• Utilized Java and multithreading concepts to ensure continuous streaming, running inside ECS containers

Datadog Metrics:

• Sent metrics and traces to Datadog, while routing logs to CloudWatch for real-time producer monitoring

Self-Healing Streaming Containers:

• Implemented self-healing streaming containers using a custom methodology
• Ensured continuous streaming by automatically restarting containers in case of technical failures, with alerts sent to Slack, Email & PagerDuty

Data Serialization and Service Development:

• Developed a generic de-serializer in Python for converting Protobuf data into JSON & Avro as part of all code activities
• Created an ALE REST API service using masked data, deployed in dev & QA regions for automated testing.

• ETL Process Design with SQL Server:
  Designed ETL processes using Informatica to load data from Flat Files, SQL server, and Excel files into the target SQL Server Data Warehouse database.
• Support and Collaboration:
  Provided support for code/design analysis, strategy development, and project planning.Collaborated with infrastructure, network, database, application, and BI teams to ensure data quality and availability.
• Spark for Interactive Queries and Streaming:
  Leveraged Spark for interactive queries, streaming data processing, and seamless integration with NoSQL databases to handle massive data volumes.Developed bespoke ETL solutions, encompassing batch processing and real-time data ingestion pipelines, utilizing Python and shell scripting to facilitate smooth data movement in and out of Hadoop.
• Hadoop Ecosystem Deployment:
  Contributed to streamlining business processes for a regional bank by developing, installing, and configuring Hadoop ecosystem components.