• Over 7 years of experience in data engineering and software development.
• Proficient in Python, Java, and SQL with expertise in designing data models and optimizing query performance.
• Skilled in architecting robust ETL/ELT pipelines for structured and semi-structured data.
• Extensive experience with Azure services, including Azure Data Factory, Azure SQL Database, Azure Databricks, and Azure.
• Synapse Analytics.
• Strong knowledge of data warehousing concepts, big data technologies, and analytics platforms.
Experienced in data processing frameworks like Apache Spark, Hadoop, and Apache Airflow, including custom DAGs.
Proven track record in leading projects and mentoring junior engineers.
Effective communicator and collaborator with cross-functional teams.
• Developed real-time data streaming pipelines for prompt data ingestion and analysis.
• Ensured data quality by implementing monitoring and troubleshooting of data integrity issues.
• Provided technical leadership and mentorship to junior data engineers, fostering a data-driven culture.
Programming languages: Python, PySpark, Java, SQL
Data Technologies: Snowflake, Dataflow, Databricks, Snowpipe, Apache Beam, BigQuery, MySQL, PostgreSQL, MongoDB, Cassandra
Cloud Platforms: Azure (Data Factory, SQL Database, Databricks, Synapse Analytics), AWS (Glue, Lambda, Redshift, S3, IAM, Step Functions, Lake Formation, Iceberg Tables, RDS)
Data Processing: Apache Spark, Hadoop, Apache Kafka, Amazon Kinesis, Airflow, Talend, Apache NiFi
DevOps & Tools: Terraform, Docker, Kubernetes, Jenkins, Git, Bitbucket, Jira, Confluence, and Azure DevOps
Visualization and Monitoring: Tableau, Power BI