Kalyan Borra

Anthem

06.2021 - 07.2022

Involved in creating new pipelines on Azure Data Factory as data ingestion, worked with Azure Databricks (Spark) for data prep using Pyspark and load the data into Azure Storage and Azure Synapse for reporting and data science team
Reading files (Parquet, CSV) from Azure Data Lake, Blobs to Data frames using PySpark
Written different API scripts in python and scheduled the jobs using a computer engine
Creating the Measures according to the business logic in Power BI Desktop using Data analysis expressions (DAX)
Responsible for creating SQL datasets for Power BI and Ad-hoc Reports
Experience in building power bi reports on Azure Analysis services for better visual presentation
Have knowledge in integrating Apache Airflow with other tools and technologies commonly used in data engineering and data science workflows, such as Apache Spark, Hadoop, and various databases
Knowledgeable in working with various Airflow components such as Operators, Sensors, Hooks, and Executors to execute tasks and interact with external systems
Have hands-on experience in Python programming language, with a strong understanding of its syntax, data structures, and object-oriented principles
Proficient in SQL databases MSSQL Server, MySQL (RDBMS), Oracle DB
Worked on Spark Data frames, Temporary Tables for transformations
Created Python notebooks on Azure Databricks for processing the datasets and loading them into Azure SQL databases.

Similar Profiles