Venkatarao Kolluri

Senior Data Engineer with 11+ years of expertise in Azure, Snowflake, AWS, ETL and Data Warehouse as well as Quality Testing. Specialized in designing scalable data architectures, optimizing cloud-based data pipelines, and automating workflows using Airflow, DBT and had experience in Application Production Support. Expertise in AWS cloud services such as S3, IAM and Lambda. Skilled in Snowflake Data Warehouse, including Snow pipe, Streams, Time Travel, Cloning and Managed large-scale data processing. Experience in ETL Development and Data Integration across AWS and Snowflake. Built robust data transformation workflows. Designed and developed ADF pipelines to orchestrate data ingestion, transformation, and movement over ADLS for HDInsight and Synapse Analytics. Integrated ADF with Azure Databricks, Azure Data Lake, Synapse Analytics, Snowflake, Blob Storage and SQL Databases for seamless data flow. Enabled incremental loads and delta processing using watermark and control tables to optimize performance. Configured Linked Services, Datasets and Triggers to automate and schedule ETL processes efficiently. Deep understanding of modern data formats including Delta Lake and Apache Iceberg. Developed and managed notebooks, orchestrated workflows using Databricks Jobs, and automated deployments using CI/CD integration with Git. Implemented robust logging, error handling and alerting mechanisms using ADF activity outputs and Azure Monitor. Implemented parameterized pipelines and dynamic content expressions for scalable and reusable data workflows. Implemented Real-time data ingestion pipelines from event sources into Kusto using Functions and ADF for high-throughput analytics and for large volumes data in Azure Data Explorer (ADX). Strong knowledge of Lakehouse architecture, including structured streaming, data versioning, and ACID transactions with Delta tables. Implemented Unity Catalog for centralized data governance across multiple workspaces, enabling fine- grained access control for tables, views and files. Implemented Delta Lake features such as ACID transactions, Time Travel, and Schema Evolution to ensure data consistency and auditability. Led a team of data engineers in designing scalable data pipelines, mentoring junior members, conducting code reviews and aligning solutions with business objectives through stakeholder collaboration. Strong knowledge of SQL, Postgres SQL and database management. Designed and optimized relational SQL databases. Experience in Data Modelling (Star/Snowflake Schema) and knowledge on Data Governance. Ensured data integrity, security and compliance. Experience in Apache Airflow, DBT, basic Python for workflow automation. Automated end-to-end ETL processes. Hands-on experience with CI/CD pipelines and Git for deployment automation. Ensured smooth integration and code deployment. Expertise in database performance tuning and optimization for Snowflake. Improved query execution efficiency. Developed scalable Snowflake architecture with dynamic schema management. Ensured cost-effective and high-performance storage. Automated data validation and transformation using Snowflake Streams and Tasks. Enhanced data quality and consistency. Designed and implemented scalable Data Vault 2.0 architecture in Snowflake, including Hubs, Links, and Satellites to support auditability, traceability, and flexible schema evolution. Developed Business Vault and reporting views from Raw Vault, enabling self-service analytics and regulatory compliance through consistent, historical, and source-agnostic data models. Experience with Role-Based Access Control (RBAC) for data security in Snowflake. Managed secure access control for multiple users. Implemented data masking and encryption for compliance with security standards. Ensured protection of sensitive data. Created optimized Snowflake queries for high-performance data retrieval. Reduced query latency and improved data processing. Designed ETL pipelines using Snowflake and DBT for seamless data integration. Improved data transformation and analytics. Implemented zero-copy cloning for efficient data testing and versioning. Enabled faster data recovery and environment replication. Designed CDC (Change Data Capture) processes for real-time data sync. Ensured accurate and timely data updates. Strong understanding of Snowflake caching mechanisms for query optimization. Improved overall system responsiveness. Experienced in Snowflake Resource Monitors to control compute cost. Implemented cost-effective resource allocation strategies. Successfully managed concurrent involvement in two projects. Proficient in functional, regression and end-to-end testing in Agile and Waterfall methodologies. Ensured data quality through comprehensive testing. Demonstrated leadership by mentoring and training junior team members to maintain scripting standards within the project. Diagnose and troubleshoot application problems, including software bugs, performance issues, and configuration errors. Utilize log files, monitoring tools, and other diagnostic resources to identify root causes. Apply strong technical skills and good business knowledge together with investigative techniques and problem-solving skills to identify and resolve issues efficiently and in a timely manner. Create and maintain apps run book documentation and knowledge base articles. Participate in post-incident reviews and contribute to the development of preventive measures. Respond and resolve application-related issues reported by internal and external users. Work with various teams to resolve application-related problems and enhance user experience. Identifying system improvement opportunities based on tracking product support requests or repetitive issues and making recommendations to development and engineering on potential solutions. Work on initiatives and continuous improvement process around proactive application health monitoring, reporting, and technical support. Involved in Enhancement and resolving the bugs.

• Trained in IBM Data stage and Quality stage tool
• Certified AWS cloud Practitioner
• Trained in RPA tool Blue Prism.