Vijay Singh Parmar
Pleasanton,CA
Summary
Experienced Data Engineering professional with 10+ years in Middleware Architecture, Big Data, ETL, Programming, and Cloud Computing. Proven leader adept at managing cross-functional teams and delivering scalable, secure data solutions. Skilled in crafting and executing detailed data models, working closely with system and software architects to ensure technical solutions are aligned with business objectives. Known for innovative problem-solving and a detail-oriented approach across all stages of database architecture.
Overview
7
7
years of professional experience
1
1
Certification
Work History
Lead Data Engineer
Nisum Technologies Inc.
05.2022 - Current
- Lead a cross-functional team of eight, managing both onshore and offshore resources. Oversee task assignment and ensure a 100% utilization rate, providing guidance and support to optimize team performance.
- Perform a wide range of tasks requiring creative solutions within broadly defined guidelines. Address complex technical challenges across multiple program areas, contributing to key organizational missions.
- Conduct feasibility studies and research analysis related to technology projects. Provide consultation in support of business programs, ensuring that technological solutions align with organizational goals.
- Facilitate agile ceremonies, design technical architectures, and monitor project milestones and deliverables. Coordinate with users, administrators, and engineers to identify and address business and technical requirements.
- Architect and develop a new codebase on the Databricks platform with full automation. Ensure high-quality implementation of scalable and efficient solutions.
- Design and implement data pipelines to store 100% of data on Azure using PySpark, Data Factory, and Smartsheet. Manage the integration of external vendor data with internal systems, ensuring compliance with data governance criteria.
- Lead Proof of Concept (POC) initiatives with external vendors to optimize media expenditures. Develop and sustain cooperative working relationships with project stakeholders to support successful outcomes.
- Advise on and contribute to the design of new system architectures, standards, and methods. Identify infrastructure system requirements, recommend technology solutions, and oversee the installation and configuration of communication infrastructure systems.
- Provide metrics on service level agreements and monitor performance against established goals. Implement process improvements based on research and analysis to enhance system efficiency and effectiveness.
- Troubleshoot, track, and conduct root cause analysis of system, database, and operational issues. Ensure timely resolution and prevent recurrence of technical problems.
Senior Data Engineer
Nisum Technologies Inc.
05.2019 - 04.2022
- Developed robust framework for ingestion and processing of large batch datasets, leveraging HDInsight, Python, and Shell scripts, resulting in 20% reduction in processing time.
- Engineered real-time data pipeline to handle approximately 500 million raw records from internal sources. Integrated technologies such as Hive, Python, HDInsight, and Azure Data Factory (ADF) to efficiently process and manage semi-structured data.
- Automated ETL workflows using Python and Shell scripts, streamlining processing of billions of rows of data and significantly reducing need for manual intervention.
- Optimized data processing by utilizing Spark within Python to handle large batch datasets, achieving 15% improvement in ingestion and processing speeds.
- Designed and implemented scalable data storage solutions using Azure Data Lake, improving data accessibility and reducing retrieval times.
- Conducted performance tuning and optimization of data processing tasks, enhancing system efficiency and reducing operational costs.
- Implemented data quality checks and validation processes, ensuring accuracy and consistency of processed data across various platforms.
Software Engineer
Nisum Technologies Inc.
03.2017 - 04.2019
- Created Python code to compute the financial costs of customer accessories.
- Achieved a 30% reduction in code size and a 35% decrease in code lines compared to the previous version.
- Employed Relic for performance testing, identifying application pain points and reducing resource usage by 15%.
- Developed and updated the codebase according to business requirements, ensuring 100% satisfaction and on-time delivery.
Education
MS - Computer Science
California State University
Chico, CA12.2016
Skills
- Python
- Shell Scripting
- SQL-PL/SQL
- Plotly-Dash
- Hadoop
- Spark
- Hive
- CA Workload Automation
- Azure Data Factory
- Azure Databricks
- Azure Key Vault
- Data Lake
- Data Quality Assurance
- HDInsight
- Data Pipeline Design
- ETL development
Certification
- PrivacyOps Certification
- AI Governance Certification
- Certified Scrum Master(CSM)
- Microsoft Certified Azure Data Engineer Associate
- Exam 70-532: Developing Microsoft Azure Solutions Certified
- Exam 70-487: Developing Microsoft Azure and Web Services Certified
- Exam 70-473: Designing and Implementing Cloud Data Platform Solutions Certified
- ITIL Intermediate Certificated Service Strategy
- MIT Artificial Intelligence: Implications for Business Strategy
- DELL BOOMI Integration Cloud Certified Developer
- Oracle PL/SQL Certified Developer
Publications
Research the Data Analysis and Processing Between MapReduce and Spark, The 2016 International Conference on Computational Science and Computational Intelligence (CSCI'16), 12/15/2016, Las Vegas, USA
Timeline
Lead Data Engineer
Nisum Technologies Inc.
05.2022 - Current
Senior Data Engineer
Nisum Technologies Inc.
05.2019 - 04.2022
Software Engineer
Nisum Technologies Inc.
03.2017 - 04.2019
MS - Computer Science
California State University
Similar Profiles
Saleem SyedSaleem Syed
Technical Lead at Nisum Technologies Inc, Client Tailored Brands IncTechnical Lead at Nisum Technologies Inc, Client Tailored Brands Inc
Bakhtiyar AliBakhtiyar Ali
Business Development Lead at Nisum TechnologiesBusiness Development Lead at Nisum Technologies
Pradeep Kumar PamidipalliPradeep Kumar Pamidipalli
QA Manager at Nisum Consulting Private LimitedQA Manager at Nisum Consulting Private Limited