Summary
Overview
Work History
Education
Skills
Certification
Timeline
TOOLS
Generic

Syed Ayaanulla Quadri

Houston

Summary

Possess strong analytical thinking, self-motivated, and an energetic approach with collaborative team skills and leadership skills. Experience in utilizing cloud-based technologies for data storage and processing. Adept in data visualization and reporting tools and have a strong understanding of data governance, data quality and data security best practices and standards.

Overview

6
6
years of professional experience
1
1
Certification

Work History

Data Engineer II

Murphy Oil USA
El Dorado, AR
07.2023 - Current
  • Contributing as a Data Engineer supporting enterprise retail data platforms, including initiatives tied to $1M–$3M capital projects, collaborating with cross-functional and contracting teams.
  • Designed and developed scalable data pipelines using Databricks (PySpark, SQL, Delta Lake) and Azure Synapse Analytics (Pipelines & Notebooks) for batch and near real-time data processing.
  • Spearheaded the processing of large-scale retail data, enabling seamless data integration and transformation across enterprise systems.
  • Designed and implemented end-to-end data pipelines orchestrating ingestion from on-prem Microsoft SQL Server (MSSQL) to Azure Data Lake Storage Gen2 (ADLS).
  • Developed and optimized queries on Microsoft SQL Server (MSSQL) using SQL Server Management Studio (SSMS) to analyze and transform data.
  • Executed and supported data migration from MSSQL to Azure Cloud, and played a key role in migrating workloads from Azure Synapse to Databricks, improving performance and scalability.
  • Developed and implemented robust data quality frameworks, including validation, reconciliation, and audit logging to ensure data accuracy and integrity.
  • Ensured all datasets consistently passed stringent data quality checks, maintaining high standards for data reliability, governance, and consistency.
  • Implemented SCD1 and SCD2 data modeling techniques using Delta Lake, enabling historical tracking and point-in-time analytics.
  • Built reusable ETL frameworks and utility-based transformations (incremental processing, deduplication, run-state logic) to standardize and optimize data pipelines.
  • Engineered complex pricing and cost data pipelines, integrating vendor, site, and zone-level data with business rules for unit pricing and promotional logic.
  • Integrated real-time data pipelines using Apache Kafka (Confluent Cloud) with Databricks, supporting streaming use cases such as fuel price and transaction data processing.
  • Worked with Delta Live Tables (DLT) and streaming architectures for near real-time data processing.
  • Designed and implemented monitoring solutions using Grafana and Prometheus to track Kafka clusters, pipeline health, and infrastructure performance.
  • Implemented automated alerting integrated with ServiceNow, enabling incident creation through structured webhook payloads.
  • Developed on-prem SQL Server integration pipelines using JDBC, enabling seamless data exchange between Databricks and operational systems.
  • Managed Databricks deployment lifecycle, handling artifact releases across Dev → QA → Prod environments.
  • Contributed to migration from Azure DevOps (ADO) to GitHub, improving version control and collaboration workflows.
  • Optimized pipeline performance by addressing cluster startup latency, job scheduling, and concurrency challenges.
  • Actively supported data scientists, data analysts, and enterprise analysts by resolving data-related queries and enabling delivery of business use cases.
  • Played a key role in KTLO (Keeping the Lights On) activities, including production support, monitoring, and incident resolution.
  • Monitored data pipelines daily, proactively identifying and resolving failures to maintain reliable data flow.
  • Troubleshot issues related to Unity Catalog permissions, schema evolution, and pipeline failures, ensuring system stability.
  • Maintained comprehensive documentation including technical design documents (TRDs), data flows, and pipeline logic.

Data Engineer

Devoir Software Solutions
Columbus, OH
01.2023 - 06.2023
  • Designed and built scalable data pipelines on AWS using Amazon S3, AWS Glue, EMR, and AWS Lambda.
  • Developed ETL workflows in Databricks (PySpark, SQL) to process and load data into Snowflake for analytics.
  • Optimized cloud infrastructure costs through instance right-sizing and efficient resource utilization.
  • Documented data pipelines, schemas, and data dictionaries to ensure data governance and lineage tracking.

Graduate Assistant

Texas A&M University
Commerce, TX
06.2021 - 12.2022
  • Served as a Database Administrator, responsible for maintaining and updating company's database.
  • Optimized user experience by fine-tuning stored procedures and SQL queries to improve data retrieval efficiency.
  • Proficient in Python, SQL, and Excel, with experience in developing and owning reporting for academic programs.
  • Able to work independently and collaboratively in a team environment, with a commitment to delivering high-quality work on time.
  • Migrated datasets and ETL workloads from On-prem database (MYSQL) to AWS Cloud.
  • Visualized data based on student certification in QuickSight.
  • Provided technical assistance to faculty members in cleaning and organizing unstructured data.
  • Collaborated with IT team to develop and implement data backup and recovery strategies.
  • Skilled in identifying procedural areas of improvement through data analysis using SQL, resulting in an 8% improvement in the profitability of the university certification program.
  • Utilized expertise in Cascade Server and WordPress to rectify and update the university website..
  • Collaborated with data analysts and business stakeholders to understand their data needs and developed PowerBI reports and dashboards to visualize data and communicate insights.

Data Engineer Intern

Knoah Solutions
06.2020 - 12.2020
  • Assisted in creating external Hive tables from the files stored in S3, and optimized Hive tables using partitions and bucketing for better Hive QL query execution.
  • Proficient in working with Spark ecosystem using Spark SQL and Scala queries on different formats like Text file, CSV file.
  • Knowledgeable in handling importing, transforming, and exporting data from various data sources including MySQL, structured, semi-structured, and unstructured data.
  • Participated in Agile software development methodologies.
  • Developed and maintained Snowflake database schemas, tables, views, and stored procedures to enable efficient querying and reporting.

Education

Master of Science - Computer Sciences

Texas A&M University
Commerce, TX
12-2022

Bachelor of Science - Information Technology

Osmania University
Hyderabad
07-2019

Skills

  • Data Engineering: ETL/ELT, Data Modeling, Data Warehousing
  • Programming: SQL, PySpark
  • Platforms: Azure Databricks, Azure Synapse Analytics
  • Data Storage: Azure Data Lake Storage Gen2 (ADLS), Microsoft SQL Server
  • Big Data: Delta Lake
  • Streaming: Apache Kafka (Confluent Cloud)
  • Monitoring: Grafana, Prometheus
  • Cloud: Microsoft Azure, AWS
  • Data Governance: Data Quality, Validation, Reconciliation
  • Version Control: Git (GitHub, Azure DevOps)

Certification

Microsoft Certified: Azure Data Engineer Associate

Databricks Certified: Data Engineer Associate

Timeline

Data Engineer II

Murphy Oil USA
07.2023 - Current

Data Engineer

Devoir Software Solutions
01.2023 - 06.2023

Graduate Assistant

Texas A&M University
06.2021 - 12.2022

Data Engineer Intern

Knoah Solutions
06.2020 - 12.2020

Bachelor of Science - Information Technology

Osmania University

Master of Science - Computer Sciences

Texas A&M University

TOOLS

  • Databricks
  • Azure Synapse Analytics
  • SQL Server Management Studio (SSMS)
  • Azure Data Lake Storage
  • Grafana
  • GitHub / Azure DevOps
  • Cascade servers
  • WordPress