Summary
Overview
Work History
Education
Skills
Timeline
Generic

Nishanka Aitha

Summary

Experienced with designing and optimizing data pipelines to ensure seamless data development, implementation, and testing of software applications. Adept at data analysis, mining, acquisition, validation, and visualization, with a proven ability to derive meaningful business insights from large datasets, both structured and unstructured data. Expert in creating efficient ETL pipelines, optimizing data processing workflows, and managing data warehouses. Utilized SQL and Python skills to create and maintain robust data architectures.

Overview

3
3
years of professional experience

Work History

Data Engineer

AlgoSpark Technologies (Walmart)
11.2024 - Current
  • Designed and implemented configurable data delivery pipeline for scheduled updates to customer facing data stores built with Scala.
  • Created and managed Spark clusters and configured high-concurrency clusters using Databricks to speed up the data preparation.
  • Enhanced data quality by performing thorough cleaning, validation, and transformation tasks.
  • Involved in Creating, Debugging, Scheduling and Monitoring jobs using Airflow for ETL batch processing to load into hive metastore for analytical processes.
  • Followed Agile methodology and participated in daily SCRUM meetings, sprint planning, showcases, and retrospectives.

Data Engineer

Anvil Solutions (Walmart)
08.2024 - 11.2024
  • Build pipelines and Data Transformations using Cloud technologies like Dataproc and GCP.
  • Designed and developed a data pipeline to collect data from multiple sources and inject it into Hadoop, Hive data lake using Spark.
  • Created Hive tables for efficient data loading, analysis and optimizing them for performance.
  • Developed Scala workflows for data extraction from cloud-based systems and applied transformations.
  • Integrated data engineering workflows with looper for continuous integration and continuous deployment (CI/CD).

Data Engineer

Tek Artisans (BCBS)
11.2023 - 07.2024
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Migrated legacy systems to modern big-data technologies, improving performance and scalability while minimizing business disruption.
  • Worked with complex SQL, Stored Procedures, Triggers, and packages in large databases from various servers.
  • Performed transformations using Python and Scala to analyze and gather the data in required format.

Data Engineer

Insightary LLC
01.2023 - 10.2023
  • Designed and implemented ETL pipelines to extract, transform, and load HEDIS data, ensuring data quality and compliance with regulatory standards
  • Worked with GCP cloud storage, Cloud Functions, BigQuery, Cloud SQL, Pub/Sub, and Dataflow.
  • Streamlined ETL processes to efficiently load data into star schemas, reducing data processing time .
  • Applied SQL queries and various tools to conduct data analysis and profiling.

Data Engineer

Ai9 Solutions
08.2022 - 12.2022
  • Involved in the design, development, and testing phases of Software Development Life Cycle (SDLC) using agile methodology to gather and analyze the requirements of the Application and Test-Driven Development (TDD).
  • Created ETL processes to extract data from various sources including on-premises databases, cloud storage, and transforming it according to business requirements, and loading it into target data warehouses.
  • · Reviewed basic SQL queries and edited inner, left, and right joins in Tableau Desktop by connecting live/dynamic and static datasets.
  • Consistently met project deadlines by effectively managing time and prioritizing tasks according to importance.
  • Analyzed proposed technical solutions based on customer requirements.
  • Developed scalable and maintainable code, ensuring long-term stability of the software.
  • Collaborated with management, internal and development partners regarding software application design status and project progress.

Education

Master of Science - Computer Science

University of Central Missouri
Warrensburg, MO
12-2022

Bachelor of Technology - Computer Science

Jawaharlal Nehru Technological University
05-2019

Skills

  • SQL, Python ,Scala
  • Big data technologies : Hive, Spark
  • Cloud technologies : GCP, AWS
  • Databases : Oracle, MySQL
  • NoSQL databases : MongoDB, HBase, Cassandra
  • Schedulers : Automic, Airflow

Timeline

Data Engineer

AlgoSpark Technologies (Walmart)
11.2024 - Current

Data Engineer

Anvil Solutions (Walmart)
08.2024 - 11.2024

Data Engineer

Tek Artisans (BCBS)
11.2023 - 07.2024

Data Engineer

Insightary LLC
01.2023 - 10.2023

Data Engineer

Ai9 Solutions
08.2022 - 12.2022

Master of Science - Computer Science

University of Central Missouri

Bachelor of Technology - Computer Science

Jawaharlal Nehru Technological University

Similar Profiles

CREATE PROFILE
Nishanka Aitha