Lalitha Nandiki
Little Elm,TX
Summary
Experienced IT professional with 3 years in the industry. Enthusiastic and inquisitive, with a strong background in big data technologies. Specialized in Apache Spark, and cloud-based ETL technologies including Azure Data Factory, Azure Databricks, Azure Data Lake Storage, and Azure Synapse Analytics. Proven ability to utilize these technologies for efficient data processing and analysis. Passionate about staying current with industry advancements and continuously expanding knowledge to deliver optimal solutions.
Overview
3
3
years of professional experience
1
1
Certification
Work History
Data Engineer
- Artificial Inventions, LLC.
06.2024 - Current
- Developed and Optimized complex ETL processes in Azure Data Factory, ensuring efficient data ingestion, transformation, and loading into Azure Data Lake Storage (ADLS) and SQL Data Warehouse.
- Leveraged ADF to orchestrate and schedule data workflows, automating data movement between on-premises and cloud-based systems.
- .Fine-tuned query performance and optimized database structures for faster, more accurate data retrieval and reporting.
- Created generic databricks notebooks for common set of activities to reduce code redundancy and improve code reusability.
- Combine and transform data from multiple tables, databases, and/or systems.
- Created resuable Databricks notebooks and templates for common data engineering tasks, streamlining the development process and improving team productivity.
- Developed and maintained CI/CD pipelines for Databricks using Azure Devops and Git, enabling automated deployement of notebooks, jobs, and clusters.
Data Engineer
- Excelon Solutions
06.2023 - 05.2024
- Developed ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Developed Pyspark and SparkSQL scripts for multiple data products based on the business logic. Created stored procedures that are helpful for performing Full loads and Incremental Loads.
- Worked with various big data file formats such as Parquet, CSV, and JSON in different scenarios of building data pipelines in Azure Data Factory and Azure Synapse Pipelines.
- Developed Data Cleaning and Data Validation scripts before applying business transformations on Azure Databricks.
- Designed and Developed a generic notebook to validate the data between source and target across multiple stages of data processing.
- Created databricks notebooks with delta format tables and implemented lake house architecture.
- Developed and maintained SQL queries in snowflake to extract, transform, and load (ETL) data.
- Collaborated with senior data engineers to optimize Snowflake queries and improve data processing performance.
- Optimized data processing by implementing efficient ETL pipelines and streamlining database design.
- Worked in Agile model during the project development.
Jr.Data Engineer
TCS - Hewlett Packard Client
06.2021 - 07.2022
- Understand and Analyze ETL requirements and identify solution elements for the enhancement requests and change requests.
- Developed stored procedures to populate fact and dimension tables within Azure SQL Data Warehouse.
- Created complex queries based on the business requirement using PySpark/SparkSQL in azure databricks.
- Built complex ETL/ELT pipelines for data processing in azure cloud using Azure Data Factory.
- Served as the primary contact for production support, overseeing the implementation of a new framework utilizing Azure Data Factory and Azure Databricks.
- Managed the migration of on-premises data from SQL Server to Azure Data Lake Store (ADLS) using Azure Data Factory (ADF).
- Created SQL views on top of the fact and dimension tables in SQL data warehouse.
- Participated in code reviews and provided constructive feedback to peers, fostering a culture of continuous improvement within the team.
Education
Master of Arts - Information Technology Management
Webster University
San Antonio, TX05.2024
Bachelor of Science - Civil Engineering
Geethanjali College of Engineering And Technology
Hyderabad, India05-2021
Skills
- Azure Databricks
- Apache Spark
- Azure Data Factory
- Extract Transform Load (ETL)
- SparkSQL
- Azure Synapse Analytics
- Power BI
- Pyspark
- Python
- SQL
- Excel
- Snowflake
- ADLS Gen 2
- Azure Data Studio
Websites
Certification
Azure Data Engineer Associate
(https://learn.microsoft.com/en-us/users/lalithanandiki-7327/credentials/33989ad241ccc982?ref=https%3A%2F%2Fwww.linkedin.com%2F)
Interests
Singing
Cooking
Badminton
Cricket
Languages
English
Timeline
Data Engineer
- Artificial Inventions, LLC.
06.2024 - Current
Data Engineer
- Excelon Solutions
06.2023 - 05.2024
Jr.Data Engineer
TCS - Hewlett Packard Client
06.2021 - 07.2022
Master of Arts - Information Technology Management
Webster University
Bachelor of Science - Civil Engineering
Geethanjali College of Engineering And Technology
Azure Data Engineer Associate
(https://learn.microsoft.com/en-us/users/lalithanandiki-7327/credentials/33989ad241ccc982?ref=https%3A%2F%2Fwww.linkedin.com%2F)
Similar Profiles
Lalitha NandikiLalitha Nandiki
Data Engineer at - Artificial Inventions, LLC.Data Engineer at - Artificial Inventions, LLC.
Samuel HemkerSamuel Hemker
Manager, Talent Solutions at Simplify Inventions, LLCManager, Talent Solutions at Simplify Inventions, LLC
Samuel HemkerSamuel Hemker
Manager, Talent Solutions at Simplify Inventions, LLCManager, Talent Solutions at Simplify Inventions, LLC