Meera Pardeshi
SHI International Corp.
Austin,USA
11
years of professional experience
Experienced with building and maintaining data pipelines to ensure seamless data flow. Utilizes advanced knowledge of big data technologies to drive data-driven decision-making. Track record of enhancing data architecture for improved performance and reliability.
Work History
Sr. Data Engineer
1 Year 6 Months- Led the design and implementation of enterprise AI solutions using Microsoft Copilot Studio, Azure OpenAI, Microsoft Fabric, and Azure AI Search, delivering multi-agent copilots that automate business workflows and improve information discovery across the organization.
- Architected and developed a secure B2B Dashboard platform using Microsoft Fabric, Azure App Services, Azure Storage, and Lakehouse technologies wrapped up in private endpoints and secure DNS providing real-time operational and business insights to internal and external stakeholders.
- Designed and implemented Retrieval-Augmented Generation (RAG) Data Ingestion architectures leveraging semantic search, vector search, LLM-based chunking, and enterprise knowledge repositories to improve response accuracy and user experience across AI applications.
- Led the integration of Microsoft Dynamics CRM, Dataverse, SharePoint, and Fabric data sources into enterprise AI solutions, enabling AI-driven sales insights while establishing governance controls to protect sensitive and PII data.
- Built Microsoft Fabric data pipelines, Lakehouses, and data integration solutions that ingest, transform, and enrich enterprise data to support analytics, knowledge management, and AI-powered applications.
- Implemented Azure security controls, deployment automation, and GitHub Actions CI/CD pipelines, including authentication, network restrictions, identity-based access management, and environment promotion processes to ensure secure and reliable solution delivery.
Data Engineer
2 Years- Developed custom ETL pipelines in Matillion to integrate diverse data sources into Snowflake data warehouse, facilitating comprehensive data analysis.
- Developed custom ETL pipelines in Matillion to seamlessly integrate data from diverse sources into Snowflake data warehouse.
- Integrated Snowpark to improve data ingestion speed, processing 2TB of data daily.
- Implemented uniform long-term contract flagging across 15+ data products using AWS and GCP, improving reporting accuracy and ensuring data governance consistency.
- Engineered data validation framework in Matillion ETL tool using Snowflake SQL stored procedures, ensuring data accuracy and reliability for informed decision-making.
- Took initiative in building Azure release pipelines, integrating 100+ files from development to production, reducing deployment time by 40%.
Data Services Engineer
1 Year 6 Months- Developed reusable Python patterns for Core Data Mesh, streamlining data access for over 30 producers and consumers, resulting in a more efficient data journey.
- Deployed Airflow DAGs to automate ETL and ingestion pipelines on GCP, improving workflow efficiency.
- Contributed to GCP Data ingestion pipelines for HSBC Securities Services, using tools like GCS, BigQuery, Airflow, and Docker.
- Developed an automated customer onboarding pipeline in Jenkins that facilitated GitHub repo creation, team setup, and cloud project updates, impacting over 20 projects and reducing onboarding time by 30%.
- Constructed data migration and validation pipeline in Python and MYSQL, migrating on-prem Hadoop data to S3 for enhanced data accessibility.
- Trained teams on GCP tools and built APIs for user dashboards, facilitating better data utilization and reporting.
- Trained teams on GCP tools, built APIs for user dashboards, and created utility libraries for table creation DDLs.
Data Engineer
2 Years 3 Months- Implemented data partitioning framework in Python to maintain large tables and improve response time for data retrieval and loading to data lake.
- Developed expertise in Apache Kafka while working on various data streaming projects.
- Handled service account setups, applied firewall rules, managed capacity/quotas, and utilized beeline on Hadoop clusters over 2 years.
- Implemented homomorphic encryption using GCP during Enterprise Engineer Program at HSBC to enhance data security.
- 2+ years of experience in creating CI/CD pipelines and performing Sqoop, File, Juniper, and CDC (Change Data Capture) Ingestions on Hadoop Clusters.
- Mentored 4 interns in Summer 2019, 2 interns in 2020, and 5 interns in 2021 on big data ingestion pipelines.
Senior Software Engineer
1 Year 1 Month- Automated ingestion processes for over 300 source systems, reducing data latency and ensuring timely updates for analytics.
- Developed detailed induction and training plans for 20+ new joiners across India, Toronto, and London regions, incorporating modern training methodologies and tools.
- Led design and implementation of scalable software solutions to enhance system performance.
- Mentored junior engineers, fostering technical growth and promoting best practices in coding standards.
- Collaborated with cross-functional teams to define project requirements and deliver high-quality software products.
Software Engineer
2 Years 4 Months- Logged and implemented 30+ development tasks using Python and SQL based on system requirements and documented best practices on Confluence to facilitate team collaboration.
- Tested troubleshooting methods, devised solutions, and documented resolutions for inclusion in knowledge base to support team effectiveness.
- Diagnosed SQL errors & QlikView generated scripts.
Education
Bachelors - Computer Science
Skills
Microsoft Fabric (Lakehouse
Warehouses
Data Pipelines
Data Factory
Fabric Data Agents)
Microsoft Azure (Azure OpenAI
Azure AI Search
App Services
Storage Accounts
Service Bus
Cosmos DB)
Microsoft Copilot Studio
Dataverse
Microsoft Dynamics CRM
AWS
GCP
Snowflake
BigQuery
Copilot Studio Agents
Fabric Data Agents
Multi-Agent Orchestration
Retrieval-Augmented Generation (RAG)
Azure AI Search
Semantic Search
Vector Search
LLM Integration
Prompt Engineering
Knowledge Management Systems
Python
SQL
Core Java
Shell Scripting
REST APIs
JSON
Microsoft Fabric Data Pipelines
Spark
Kafka
Apache Airflow/Composer
Matillion
ETL/ELT Development
Data Modeling
Data Integration
Lakehouse Architecture
Hive
MySQL
GitHub Actions
Azure DevOps
CI/CD Automation
Identity & Access Management (IAM)
Authentication & Authorization
Network Security
Private Endpoints
Data Governance
Timeline
Sr. Data Engineer
SHI International Corp.
02.2025 - CurrentRead More
Data Engineer
N-able
02.2023 - 02.2025Read More
Data Services Engineer
HSBC
08.2021 - 02.2023Read More
Data Engineer
HSBC
05.2019 - 08.2021Read More
Senior Software Engineer
HSBC
04.2018 - 05.2019Read More
Software Engineer
HSBC
11.2015 - 03.2018Read More
Government College of Engineering And Research
Bachelors from Computer Science
Read More