Manasija Lingampally
Kansas City,MISSOURI
Summary
Skilled Associate Data Engineer with background in designing, building and maintaining data processing systems. Familiarity with identifying patterns and trends in large datasets adds to ability to deliver innovative solutions for complex business challenges. Demonstrated strengths include teamwork, problem-solving skills, and proficiency in SQL, Python, and Hadoop. Previous roles have involved enhancing data collection procedures to improve overall data reliability and quality. Resourceful Associate Data Engineer known for high productivity and efficient task completion. Specialize in data pipeline development, database management, and big data technologies. Excel in problem-solving, teamwork, and adaptability, ensuring smooth project execution and innovative solutions to complex challenges.
Overview
4
4
years of professional experience
Work History
Data Engineer
Wells Fargo
Chapel Hill, NC
01.2024 - Current
- Designed and developed ETL pipelines to extract data from various sources, transform it using SQL queries and Azure Data Factory (ADF), and load it into Azure SQL Database or other target databases.
- Created ETL scripts to automate manual processes for efficient data loading.
- Generated reports based on analytical findings that enable stakeholders to make informed decisions.
- Performed quality assurance checks on incoming datasets, identifying and resolving issues with accuracy and precision.
- Designed, constructed, and maintained scalable data pipelines for data ingestion, cleaning, and processing using Python and SQL.
- Conducted data analysis using SQL and Python to derive insights and support decision-making processes.
- Utilized SQL to query large datasets from multiple sources.
- Developed and maintained data pipelines to ingest, transform, validate, and store data in a variety of formats.
- Utilized Apache Spark to process large data sets for analysis.
- Implemented best practices while developing ETL packages like error handling, logging.
- Extracted data from diverse sources such as SQL databases (SQL Server, MySQL), flat files (CSV, JSON), and cloud storage (Azure Blob Storage, AWS S3) using Azure Data Factory pipelines.
- Integrated third-party applications with Microsoft Azure using APIs and RESTful web services.
- Stayed current with Azure services and technologies, applying best practices to ongoing projects.
- Managed Azure IaaS and PaaS resources such as SQL Database, Service Bus Queues, Event Hubs and Automation Accounts.
Data Engineer
Charter Communications (spectrum )
Dallas, TX
08.2022 - 01.2023
- Conducted research on emerging technologies related to Big Data processing, storage, and analytics.
- Integrated existing systems with new platforms such as AWS S3.
- Developed Python scripts to automate data analysis tasks.
- Integrated AWS services with third party tools such as Jenkins CI and CD pipelines for automated deployments across multiple environments.
- Implemented serverless architectures with AWS Lambda functions, API Gateway and DynamoDB.
- Configured storage solutions with Amazon Simple Storage Service buckets to store static content like images, videos.
- Developed Python scripts to automate data processing tasks.
- Evaluated various libraries and frameworks to identify best solutions for specific requirements.
- experience in using Pandas for data manipulation tasks such as filtering, transforming, and aggregating large datasets efficiently.
- Applied Pandas for data transformation tasks such as reshaping dataframes, merging/joining datasets, and creating new variables/columns.
- Managed data storage solutions on AWS, including setting up and configuring RDS (Relational Database Service) and DynamoDB for NoSQL databases.
Data Engineer
Meta more solutions
Hyderabad, Telangana
08.2020 - 11.2021
- Provided technical support regarding database management systems such as Oracle or MySQL.
- Created database objects such as tables, views, stored procedures, triggers. in SQL Server, MySQL, Oracle environments.
- Created ETL scripts to automate manual processes for efficient data loading.
- Developed and deployed machine learning models for predictive analytics, utilizing Spark and TensorFlow.
- Designed, constructed, and maintained scalable data pipelines for data ingestion, cleaning, and processing using Python and SQL.
- Compiled reports based on collected data using MS Excel or other software programs.
- Store the resultant processed data back into Hadoop Distributed File System.
- Implemented data processing tasks using MapReduce jobs to analyze large datasets distributed across Hadoop clusters, gaining insights through distributed computing paradigms.
- Documented ETL processes, data mappings, and Hadoop cluster configurations. Collaborated with team members to streamline workflows and improve data processing efficiency.
- Loaded data from MySQL server to the Hadoop clusters using the data ingestion tool Sqoop.
- Developed Spark code using Scala and Spark Streaming for faster testing and processing of data.
Education
Master of Science - Big Data Analytics
University of Central Missouri
Warrensburg, MO05-2023
Bachelor of Science - Electrical, Electronics Engineering Technologies
Matrusri Engineering College
Hyderabad06-2021
Skills
- Hadoop
- PL/SQL
- Spark
- AWS (S3,Lambda)
- Data Migration
- Data Warehousing
- Data Modeling
- Scripting Languages
- Sqoop
- Linux
- Microsoft Azure
- Python Programming Language
- Python and C
- Programming Languages: Java, C#, Net, Python
- SQL Server
- Oracle PL/SQL
- MySQL
- ETL importing
Project Details
Title: Rapid document conversion April 2020
Description: By using AWS Lambda, we have developed an app that can rapidly convert documents from one format to the other. we can retrieve the required content and can format and convert the content to download or display on the webpage.
Title: Serverless Webapp Nov 2019
Description: With the help of AWS we have developed a serverless web app that is secure and usable with the help of AWS (Amplify, Lambda).By using AWS Cognito we have managed the authentication and administration for the back-end.
Title: IOT based smart traffic management system June 2019
Description: An IOT based traffic management solutions for smart cities where traffic flow can be dynamically controlled. An additional time is provided based on the basis of Traffic flow which prevents traffic clogging and improves Time efficiency. Traffic is controlled on the basis of density count
Tittle: HealthCare System Big Data Analytics May 2022
Description: •The goal of the project is to create ETL data pipelines for the Health Care insurance company which will make the company make appropriate business strategies to enhance their revenue by analyzing customers behaviors and send offers and royalties to customers respectively.
Tools: Linux, Hadoop, Spark, Sqoop, Python.
Timeline
Data Engineer
Wells Fargo
01.2024 - Current
Data Engineer
Charter Communications (spectrum )
08.2022 - 01.2023
Data Engineer
Meta more solutions
08.2020 - 11.2021
Master of Science - Big Data Analytics
University of Central Missouri
Bachelor of Science - Electrical, Electronics Engineering Technologies
Matrusri Engineering College