Pavan Kondepudi

Experienced Enterprise Data Architect with solid background in designing, creating, and managing large-scale data architectures. Strengths include thorough knowledge of database structures, systems planning, and implementation coupled with skills in strategic data analysis and modeling. Known for effective leadership in directing teams towards successful project completion while ensuring optimal data integrity and system security. Previous roles have resulted in improved efficiency through streamlining processes and implementing innovative solutions within the enterprise architecture framework.

• 10+ years of IT experience in Big Data/Hadoop/Azure Development, with special expertise in System Analysis, Architecture, Design, Development, and Implementation.
• Project Management, Project Estimation, Cost Control, Cost Management, and Cost Planning.
• Collaborate with Scrum Team, Project Managers, and business owner to develop the application.
• Hands-on experience using Hortonworks Distribution of Hadoop (2.4 and 2.6) and Microsoft Azure.
• Extensive experience in working with HDFS, MapReduce, Pig, Hive, Sqoop, Flume, Oozie, Zookeeper, Kafka, Spark, HUE, HBase, Ambari, and Phoenix.
• Experience in data processing, ingestion, storage, querying, and analysis using Spark SQL and Spark Streaming.
• Design, implement, and manage operations framework (security, automation, incident management, monitoring, and notification) for big data applications.
• Provide design and technical mentorship to teams and support management in identifying and implementing competency development measures.
• Experience in Hadoop On-prem to Microsoft Azure migration project.
• Experienced in Microsoft Azure tools (ADLS, Synapse Analytics, Spark Pool, and Synapse Analytics Workspace).
• Assist the development team in identifying the root cause of slow-performing jobs and queries.
• Developed data pipelines using Spark for real-time streaming to store data into HDFS, Hive, and HBase from Kafka.
• Experience in tuning performance by using Partitioning and Bucketing in Hive.
• Experience on Talend Open Studio 6.4 and Talend Administration Center (TAC), GitHub, and Jenkins.
• Experience in implementing Spark using Scala to perform complex data transformations and converting Hive/SQL queries into Spark transformations using Scala.
• Worked with Spark Streaming and Kafka to load the sensor event data into HDFS.
• Developed Spark code and Spark-SQL/Streaming for faster testing and processing of data.
• Worked on using different file formats like Sequence, AVRO, ORC files, Parquet files, and CSV using different compression techniques.
• Experience in importing and exporting data from various databases like Teradata, Oracle, and DB2 into HDFS using Sqoop.
• Experience in creating subscriptions in IBM InfoSphere Data Replication (IIDR) to flow data from DB2 and Oracle to Kafka.
• Loading data from different sources (database and files) into Hive, Kafka using Talend tool.
• Experience in developing MapReduce programs using Java and Scala.
• Knowledge of scheduling and monitoring tools like Oozie, Control-M, and Zena.
• An excellent team player and self-starter with good communication skills and proven abilities to finish tasks before target deadlines.
• Extensive knowledge in Software Development Life Cycle, along with software methodologies like Waterfall and Agile.
• Expertise in Unit Testing, Functional Testing, System Testing, Integration Testing, Regression Testing, User Acceptance Testing (UAT), and Performance Testing.
• Excellent communication, analytical skills, and flexibility to learn new tools and technologies to meet the needs of the organization towards the company's success.

Microsoft Azure Solutions Architect