Kalyan Yalagandula
Summary
13 years of experience in the design, development, and implementation of robust technology systems, with specialized expertise in Snowflake Schema Admin Hadoop Administration and Linux Administration. A certified SnowPro Administrator with 6 years of hands-on experience in Snowflake Administration, specializing in data modeling, query optimization, and managing Snowflake environments on the cloud. 5 years of experience in Hadoop Administration and Big Data Technologies, with expertise in managing and optimizing Hadoop clusters on cloud platforms like Azure (with Insight), Google Cloud, and AWS. 4 years of experience in Linux Administration, specializing in system configuration, performance tuning, troubleshooting, and ensuring optimal server performance. Experience with complete Software Design Lifecycle including design, development, testing, and implementation of moderate to advanced complex systems. Hands-on experience in designing, implementing, and managing Snowflake schemas in data warehousing environments. Experience in data modeling and ETL processes in Snowflake schema, including fact and dimension tables, ensuring efficient query performance and scalability. Expertise in managing Snowflake cloud data platform including user access, roles, and data security configurations. Proficient in performing performance tuning for Snowflake queries, utilizing clustering keys and materialized views for optimization. Experience in automating ETL workflows using Snowflake tasks and streams, ensuring timely data updates for reporting and analysis. Ability to work with semi-structured and structured data in Snowflake using VARIANT, OBJECT, and ARRAY data types. Proficient in managing Snowflake’s data sharing capabilities and sharing secure, governed datasets with external partners. Experience in Snowflake storage management, including managing virtual warehouses, scaling compute resources, and optimizing cost efficiency. Hands-on experience in troubleshooting and monitoring Snowflake performance, ensuring high availability and fast query execution. Familiar with integrating Snowflake with third-party tools like ETL tools (e.g., Talend, Informatica), BI tools (e.g., Tableau, Power BI), and other data lakes. Experience in setting up and managing Snowflake’s data governance practices, including auditing, logging, and compliance measures. Proficient in using SQL for querying and managing large datasets within Snowflake, as well as handling complex queries involving joins and subqueries. Expert in Snowflake Data Replication and Failover configurations to ensure disaster recovery and business continuity.
Overview
15
15
years of professional experience
Work History
Sr. Snowflake Admin / Support Engineer
State Farm
Az
04.2020 - Current
- Managed the migration process from Hadoop's MapReduce to Snowflake's native SQL processing, ensuring minimal downtime and preserving data integrity during the migration
- Performed detailed data analysis and profiling of Hadoop-based datasets before migration, identifying data quality issues and ensuring accurate data transformation to the Snowflake environment
- Managed Snowflake user accounts, roles, and permissions, ensuring secure access controls and compliance with organizational policies
- Implemented multi-factor authentication (MFA) and role-based access controls (RBAC) to safeguard sensitive data and ensure secure operations
- Established and enforced data governance frameworks, ensuring data security, integrity, and compliance with regulatory standards
- Monitored and audited user activity within Snowflake to ensure adherence to internal security protocols and compliance regulations
- Optimized query performance by managing virtual warehouses, using clustering keys, and implementing materialized views
- Conducted regular performance analysis, identifying bottlenecks and implementing best practices to enhance query execution and system performance
- Integrated Snowflake with various ETL tools for seamless data migration and transformation across cloud platforms
- Automated data loading and processing using Snowflake Streams and Tasks, ensuring real-time data availability and efficient processing
- Designed and maintained Snowflake schemas (Star, Snowflake) for efficient data storage, ensuring high-performance query capabilities
- Managed data sharing processes, providing secure, governed access to stakeholders and external partners
- Monitored Snowflake usage and cost metrics, adjusting warehouse sizes, scaling strategies, and auto-suspend features to optimize resource consumption and control operational costs
- Implemented cost-saving practices, including efficient resource allocation, scaling virtual warehouses dynamically based on workload demand
- Identified, diagnosed, and resolved Snowflake performance issues, ensuring high availability and minimizing downtime
- Provided ongoing support for Snowflake users, troubleshooting query failures, data access issues, and ensuring optimal system performance
- Install, configure, and maintain GoldenGate for real-time data replication
- Design and implement high-availability and disaster recovery solutions
- Monitor and troubleshoot replication lag, conflicts, and performance issues
- Optimize data synchronization between heterogeneous databases like Oracle, SQL Server, and Snowflake
- Extensive experience working with AWS, Azure, and other cloud platforms for data migration and integration
- Deep knowledge of Data Migration strategies, ETL processes, and Cloud Architecture best practices
- Expertise in Data Security measures, ensuring data protection and compliance with regulatory standards
- Environment: Hadoop, Snowflake, SQL, AWS, Azure, Data Migration, ETL, Data Integration, Cloud Architecture, Data Security, MFA
Snowflake Admin & support engineer
Walmart Labs INC
AR
01.2019 - 03.2020
- Managed the migration process from Oracle databases to Snowflake for data storage and processing in Azure, ensuring seamless integration with minimal downtime and no data loss
- Performed comprehensive data analysis and profiling of Oracle-based datasets, identifying data quality issues and ensuring proper transformation into Snowflake’s cloud architecture
- Managed Snowflake user accounts, roles, and permissions, enforcing secure access controls and ensuring compliance with organizational data governance policies
- Implemented multi-factor authentication (MFA) and role-based access control (RBAC) to ensure secure user access and protect sensitive data within Snowflake
- Established data governance frameworks, ensuring compliance with industry standards, regulatory requirements, and organizational security protocols
- Monitored and audited user activities within Snowflake, identifying and addressing any deviations from internal security policies
- Optimized query performance by configuring virtual warehouses, implementing clustering keys, and utilizing materialized views to improve query execution times
- Integrated Snowflake with various ETL tools to enable efficient data migration, processing, and transformation across different cloud environments
- Automated data loading and processing through Snowflake Streams and Tasks, ensuring real-time data availability for analysis and reporting
- Designed and maintained Snowflake schemas (Star, Snowflake) to optimize data storage and ensure high-performance querying
- Monitored Snowflake usage and costs, dynamically adjusting warehouse sizes and implementing auto-suspend features to control resource consumption and minimize costs
- Implemented cost-saving strategies, including efficient resource allocation and scaling virtual warehouses based on workload demand
- Troubleshot Snowflake performance issues, identifying and resolving bottlenecks to maintain system uptime and performance
- Provided ongoing support for Snowflake users, resolving issues related to query failures, data access, and system performance
- Configured and maintained Oracle GoldenGate for real-time data replication between Oracle, Snowflake, and other cloud databases
- Designed and implemented high-availability and disaster recovery solutions, ensuring data redundancy and system uptime
- Monitored replication processes, addressing replication lag, conflicts, and performance issues for smooth data synchronization between heterogeneous databases
- Environment: Oracle, Snowflake, Azure, AWS, Data Migration, ETL, Cloud Architecture, Data Security, MFA, RBAC
Sr. Software Engineer (Hadoop infrastructure analyst)
State Farm
Tempe, AZ
01.2018 - 12.2018
- Experienced as an admin on Cloudera (CDH 5.13.2) distribution for 12 clusters ranging from POC to PROD
- And Horton Works (2.5.3)
- Responsible for Cluster maintenance, Monitoring, commissioning, and decommissioning Data nodes, Troubleshooting, Managing and reviewing data backups, and managing & and reviewing log files
- Day-to-day responsibilities include solving developer issues, deployments moving code from one environment to another environment, providing access to new users and providing instant solutions to reduce the impact of documenting the same, and preventing future issues
- Adding/installation of new components and removal of them through Cloudera Manager
- Collaborating with application teams to install the operating system and Hadoop updates, patches, and version upgrades
- Designed and managed Apache Airflow DAGs for automated job scheduling and failure recovery
- Worked on the journal node sync and Name node high availability and yarn, hue, and oozie, and Hbase and impala high availability
- Experienced in Ranger and Atlas integration
- And provide the atlas and ranger role-based policies
- Performance tuning on Hue and Hive, impala, and Spark jobs
- Monitored workload, job performance, and capacity planning using Cloudera Manager
- Interacting with Cloudera support logging the issues in the Cloudera portal and fixing them as per the recommendations
- Cluster provisioning depends upon the user request and adding gateway nods to the cluster (CRMT tool)
- Retrieved data from HDFS into relational databases with Sqoop
- Parsed cleansed and mined useful and meaningful data in HDFS using Map-Reduce for further analysis
- Implemented custom interceptors for flume to filter data and defined channel selectors to multiplex the data into different sinks
- Partitioned and queried the data in Hive for further analysis by the BI team
- Involved in extracting the data from various sources into Hadoop HDFS for processing
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database, and Sqoop
- Sentry installation and configuration
- Experienced in using Kafka as a distributed publisher-subscriber messaging system
- Creating and truncating Hbase tables in hue and taking backup of submitter ID(s)
- Configuring and managing permissions for the users in hue
- Responsible for building scalable distributed data solutions using Hadoop
- Commissioned and Decommissioned nodes on CDH5 Hadoop cluster on Redhat LINUX
- Involved in loading data from the LINUX file system to HDFS
- Implemented test scripts to support test-driven development and continuous integration
- Worked on tuning the performance of Pig queries
- Worked with application teams to install operating system, Hadoop updates, patches, and version upgrades as required
- Experience in configuring the Storm in loading the data from MYSQL to HBASE using JMS
- Responsible for managing data coming from different sources
- Involved in loading data from UNIX file system to HDFS
- Experience in managing and reviewing Hadoop log files
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
- Installed Oozie workflow engine to run multiple Hive and Pig jobs
- Analyzed large amounts of data sets to determine the optimal way to aggregate and report on them
- Supported in setting up the QA environment and updating configurations for implementing scripts with Pig and Sqoop
- Environment: HDFS, Map Reduce, Hive, Hue, Pig, Flume, Oozie, Sqoop, Cdh5, Apache Hadoop Spark, Solr, Storm, Cloudera Manager, Red hat, MySQL, and Oracle
Sr. Hadoop Operations & Administrator
All State
IL
04.2017 - 12.2017
- Experienced as admin in Horton works (HDP 2.5.1) distribution for 5 clusters ranging from POC to PROD
- Cluster capacity planning depends upon the data usage
- Designed and configured the Baston-Edge node configuration
- Designed and configured HA of Hive & Hbase services
- Fine-grained authorization to do a specific action and/or operation with a Hadoop component/tool and managed through a central administration Ranger tool
- Experienced in managing and reviewing log files
- (Identify the max backup index and max backup size of Log4j properties of all services in Hadoop) and integrated the Hadoop log4j properties of AWS Cloud Watch
- Experienced on use to dump the data from Hadoop to AWS S3 buckets through the access key and secret key
- Experienced on a setup spark service on AWS EMR cluster on EC2 instances (S3 nodes)
- Changed the zookeeper and journal node edit directories (zookeeper and journal nodes have multiple directories)
- Responsible for Cluster maintenance, Monitoring, commissioning, and decommissioning Data nodes, Troubleshooting, Managing and reviewing data backups, and managing & and reviewing log files
- Day-to-day responsibilities include solving developer issues, deployments moving code from one environment to another environment, providing access to new users providing instant solutions to reduce the impact documenting the same, and preventing future issues
- Experienced in adding/installing new components and removal of them through Ambari
- Monitoring systems and services through the Ambari dashboard to make the clusters available for the business
- Architecture design and implementation of deployment, configuration management, backup, and disaster recovery systems and procedures
- Hand-on experience on cluster upgradation and patch upgrade without any data loss and with proper backup plans
- Changing the configurations based on the requirements of the users for the better performance of the jobs
- Experienced in Ambari-alerts (critical & and warning) configuration for various components and managing the alerts
- Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster
- Good troubleshooting skills on Hue, which provides GUI for developers/business users for day-to-day activities
- Developed Map Reduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis
- Implemented complex Map Reduce programs to perform joins on the Map side using distributed cache
- Implemented Name Node HA in all environments to provide high availability of clusters
- Involved in snapshots and mirroring to maintain the backup of cluster data and even remotely
- Experienced in managing and reviewing log files
- (Identify the max backup index and max backup size of Log4j properties of all services in Hadoop)
- Helping the users in production deployments throughout the process
- Experienced in production support which involves solving user incidents varies from sev1 to sev5
- Managed and reviewed Log files as a part of administration for troubleshooting purposes
- Communicate and escalate issues appropriately
- As an admin followed standard Backup policies to ensure the high availability of the cluster
- Documented the systems processes and procedures for future reference
- Worked with the systems engineering team to plan and deploy new environments and expand existing clusters
- Monitored multiple cluster environments using AMBRI Alerts, Metrics
- Environment: HDFS, Yarn, Map reduce, pig, zookeeper, spark, Kafka, Horton works, Ambari
Software Engineer (Hadoop)
Synchrony Financial
Chicago, IL
06.2016 - 03.2017
- Experienced as admin in Horton works (HDP 2.2.4.2) distribution for 6 clusters ranging from POC to PROD
- Responsible for Cluster maintenance, Monitoring, commissioning, and decommissioning Data nodes, Troubleshooting, Managing, and reviewing data backups, and managing & and reviewing log files
- Day-to-day responsibilities include solving developer issues, and deployments moving code from one environment to another environment, providing access to new users, and providing instant solutions to reduce the impact of documenting the same and preventing future issues
- Experienced in adding/installing new components and removal of them through Ambari
- Monitoring systems and services through the Ambari dashboard to make the clusters available for the business
- Architecture design and implementation of deployment, configuration management, backup, and disaster recovery systems and procedures
- Hand-on experience on cluster upgradation and patch upgrade without any data loss and with proper backup plans
- Changing the configurations based on the requirements of the users for the better performance of the jobs
- Experienced in Ambari-alerts configuration for various components and managing the alerts
- Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster
- Experience in Standardize authorization Ranger Acees method across all Hadoop components
- Good troubleshooting skills on Hue, which provides GUI for developers/business users for day-to-day activities
- Developed Map Reduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis
- Implemented Name Node HA in all environments to provide high availability of clusters
- Capacity scheduler implementation in all environments to provide resources based on the allocation
- Create queues and allocate the cluster resources to provide priority for jobs
- Experienced in Setting up the project and volume setups for the new projects
- Involved in snapshots and mirroring to maintain the backup of cluster data and even remotely
- Implementing the SFTP for the projects to transfer data from External servers to servers
- Experienced in managing and reviewing log files
- Working experience in maintaining MySQL database creation setting up the users and maintaining the backup of cluster metadata databases with corn jobs
- Setting up MySQL master and slave replications and helping business applications maintain their data in MySQL Servers
- Helping the users in production deployments throughout the process
- Experienced in production support which involves solving user incidents varies from sev1 to sev5
- Managed and reviewed Log files as a part of administration for troubleshooting purposes
- Communicate and escalate issues appropriately
- As an admin followed standard Backup policies to ensure the high availability of the cluster
- Involved in Analyzing system failures, identifying root causes, and recommending courses of action
- Documented the systems processes and procedures for future reference
- Worked with the systems engineering team to plan and deploy new environments and expand existing clusters
- Monitored multiple cluster environments using AMBRI Alerts, Metrics, and Nagios
- Environment: Hadoop HDFS, Map Reduce, Hive, Pig, Flume, Oozie, Sqoop, Eclipse, Horton works, and Ambari
Sr. Linux System Engineer
Cigniti Technologies
HYD, IND
09.2012 - 04.2014
- Installation and configuration of Linux for the new build environment
- Created a Virtual server on Citrix Xen server-based host and installed operating system on Guest Servers
- Configuring NFS, DNS
- Updating YUM Repository and Red Hat Package Manager (RPM)
- Created RPM packages using RPMBUILD, verifying the new build packages and distributing the package
- Configuring distributed file systems administering NFS server and NFS clients and editing auto-mounting mapping as per system/user requirements
- Installation, configuration, and maintenance of FTP servers, NFS, RPM, and Samba
- Configured SAMBA to get access to Linux-shared resources from Windows
- Created volume groups logical volumes and partitions on the Linux servers and mounted file systems on the created partitions
- Deep understanding of monitoring and troubleshooting mission-critical Linux machines
- Experience with Linux internals, virtual machines, and open-source tools/platforms
- Improve system performance by working with the development team to analyze, identify, and resolve issues quickly
- Ensured data recoverability by implementing system and application-level backups
- Performed various configurations which included networking and Iptables, resolving hostnames, and SSH keyless login
- Managing Disk File Systems, Server Performance, user creation, and Granting file access Permissions and RAID configurations
- Support pre-production and production support teams in the analysis of critical services and assist with maintenance operations
- Automate administration tasks using scripting and Job Scheduling using CRON
- Environment: Linux, Citrix Xen Server 5.0, veritas volume manager, and net backup
Linux Administrator
People Link
HYD, IND
01.2010 - 08.2012
- Patched RHEL5 and Solaris 8, 9, and 10 servers for EMC Power Path Upgrade for VMAX migration
- Configured LVM (Logical Volume Manager) to manage volume group, logical and physical partitions, and importing new physical volumes
- Maintained and monitored all servers' operating system and application patch level, disk space and memory usage, user activities daily, administration on Sun Solaris and RHEL systems, and management archiving
- Installed, configured, troubleshooted, and maintained Linux Servers and Apache Web servers, configured and maintenance of security and scheduling backups, and submitted various types of corn jobs
- Installations of HP Open view, a monitoring tool, in more than 300 servers and worked with monitoring tools such as Nagios and HP Open view
- Creation of VMs, cloning, and migrations of the VMs on VMware vSphere 4.0/4.1
- Setup and configured Apache to integrate with IBM WebSphere in load balancing environment
- RHEL 4.1, Red Hat Linux 8/9, IBM x series and HP ProLiant, Windows
- Installing and upgrading OE & Red Hat Linux and Solaris 8/9/10 x86 & SPARC on Servers like HP DL 380G3, 4, and 5 & and Dell Power Edge servers
- Environment: Linux 5/4, Sun Solaris 10/9/, Oracle 10g, SUN Servers, SUN Fires, Linux, HP open view service desk (OVSD), Kick start, Jump Start, Fujitsu Prime power servers, Samba
Education
Bachelor’s - computer science
Kakatiya University
IndiaMS - software engineering
International Technological University
USASkills
- Data modeling
- Query optimization
- Snowflake environment management
- Performance tuning
- Security configuration
- Cloud integration
- Map Reduce
- HDFS
- Pig
- Hive
- HBase
- Sqoop
- Zookeeper
- Oozie
- Hue
- Storm
- Kafka
- Spark
- Flume
- Java
- Core Java
- HTML
- Programming C
- C
- PL/SQL
- Linux (RHEL, Ubuntu)
- Open Solaris
- AIX
- Shell Scripting
- HTML scripting
- Aws
- Google Cloud Platform
- Microsoft Azure
- HDP Ambari
- Cloudera Manager
- Oracle SQL
- PostgreSQL
- NoSQL (MongoDB, Cassandra)
Timeline
Sr. Snowflake Admin / Support Engineer
State Farm
04.2020 - Current
Snowflake Admin & support engineer
Walmart Labs INC
01.2019 - 03.2020
Sr. Software Engineer (Hadoop infrastructure analyst)
State Farm
01.2018 - 12.2018
Sr. Hadoop Operations & Administrator
All State
04.2017 - 12.2017
Software Engineer (Hadoop)
Synchrony Financial
06.2016 - 03.2017
Sr. Linux System Engineer
Cigniti Technologies
09.2012 - 04.2014
Linux Administrator
People Link
01.2010 - 08.2012
Bachelor’s - computer science
Kakatiya University
MS - software engineering
International Technological University
Similar Profiles
ALBERTO ESCOBAR RUIZALBERTO ESCOBAR RUIZ
Claim Specialist; and Producer at State Farm Corporate & State Farm AgencyClaim Specialist; and Producer at State Farm Corporate & State Farm Agency
Bethany GandyBethany Gandy
Insurance Customer Service Specialist at Brian Moore State Farm & Brent Holman State FarmInsurance Customer Service Specialist at Brian Moore State Farm & Brent Holman State Farm
Dedra ClayDedra Clay
Underwriting Service Assistant Team Resource at State FarmUnderwriting Service Assistant Team Resource at State Farm
Isaiah SkinnerIsaiah Skinner
Claims Adjuster at State FarmClaims Adjuster at State Farm
Qais F. AlmaghairehQais F. Almaghaireh
Sr. IT Support Engineer / O365 Admin / SD Admin at Al Ansari Exchange LLCSr. IT Support Engineer / O365 Admin / SD Admin at Al Ansari Exchange LLC