KARTHIK REDDY KARNA

To secure a challenging role as a Data Engineer, leveraging 9+ years of software industry experience, with a focus on Azure cloud services and Big Data technologies like Spark, MapReduce, Hive, Yarn, and HDFS, using programming languages such as Scala and Python. With 4 years of experience in Data Warehouse, I possess a deep understanding of ETL processes, data modeling, and data warehousing. I am committed to delivering efficient and scalable data solutions that drive business growth and support strategic decision-making.

• Highly results-driven Data Engineer specializing in the design and implementation of scalable data ingestion pipelines using Azure Data Factory.
• Expertise in leveraging Azure Databricks and Spark for distributed data processing and transformation tasks, ensuring optimal performance.
• Skilled in maintaining data quality and integrity through robust validation, cleansing, and transformation operations.
• Adept at architecting cloud-based data warehouse solutions on Azure, utilizing Snowflake for efficient data storage, retrieval, and analysis.
• Extensive experience with Snowflake Multi-Cluster Warehouses and deep understanding of Snowflake cloud technology.
• Proficient in utilizing advanced Snowflake features such as Clone and Time Travel to enhance database applications.
• Actively involved in the development, improvement, and maintenance of Snowflake database applications.
• Expertise in building logical and physical data models for Snowflake, adapting them to meet changing requirements.
• Skilled in defining roles and privileges to control access to different database objects.
• Thorough knowledge of Snowflake database, schema, and table structures, enabling efficient data organization and retrieval.
• Strong collaboration skills, working closely with data analysts and stakeholders to implement effective data models and structures.
• Proven proficiency in optimizing Spark jobs and utilizing Azure Synapse Analytics for big data processing and advanced analytics.
• Track record of success in performance optimization and capacity planning to ensure scalability and efficiency.
• Experienced in developing CI/CD frameworks for automated deployment of data pipelines, collaborating with DevOps teams.
• Proficient in scripting languages such as Python and Scala, enabling efficient automation and customization.
• Skilled in utilizing Hive, SparkSQL, Kafka, and Spark Streaming for ETL tasks and real-time data processing.
• Strong working experience in Hadoop ecosystem, including HDFS, Map-Reduce, Hive, and Python.
• Hands-on expertise in developing large-scale data pipelines using Spark and Hive.
• Experience in using Apache Sqoop for importing and exporting data between HDFS and relational databases.
• Proven experience in setting up and managing Hadoop job workflows using Apache Oozie.
• Optimized query performance in Hive using advanced techniques like bucketing and partitioning, and extensive Spark job tuning experience.
• Highly proficient in Agile methodologies, utilizing JIRA for efficient project management and reporting.