DIGAMBER KALIVEMULA
BIRMINGHAM,AL
Summary
Background includes warehousing and analytics. Proficient in machine and deep learning. Quality-driven and hardworking with excellent communication and project management skills.
Overview
3
3
years of professional experience
Work History
DATA ENGINEER
314e India Corporation
01.2020 - 01.2022
- Built an ETL project which involves enabling systems to ingest different standards of healthcare data(FHIR, SQL tables, CCDA) and converting it into hl7V2 standard
- Worked on IBM open source project called FHIR server.Built API's on top of IBM FHIR server and integrated it with docker to support large scale data access
- Implemented audit log systems to monitor the data access and created a intrusion detection system to log and report any unauthorised access
- Implemented REST API's using fastAPI, tested projects built using Django
- Designed and developed API's which work in data-bricks environment and improved the performance over the time.
SOFTWARE DEVELOPER/ETL DEVELOPER
314e India Corporation
01.2019 - 01.2020
- Successfully created a ETL project to move data from RDBMS to data-bricks and architected data base, committing to database design principles (ACID properties and Normalisation)
- Enhanced performance of various parts of data warehouse(storing and retrieving data) and ETL pipe- lines(Processing and manipulating data) using data compression techniques, query optimisation and in memory processing
- Improved data retrieving methods using query optimisation using Apache spark for the better utilisation of resources and faster processing
- Which improved overall efficiency by 68%
- Applied various compression methods and in memory processing techniques like pyArrow and worked with multiple file formats like parquet and Avro to implement modules for faster data access and data manipulation
- Through which storage cost was reduced to 50% and the access time was decreased by a factor of 1.5
- Wrote complex SQL queries to analyse real-time batch data and created a dash board to visualise the results.And performed advanced sql query performance tuning to improve programme efficiency
- Created testing environment using containers in docker
- Automated the unit testing process for the projects updates and new releases
- Derived and designed product features and tools from regular customer interaction.
Education
Master's - Data Science
University of Alabama
Bachelor of Technology - Computer Science and Engineering
VNR Vignana Jyothi Institute of Engineering & Technology
2015
Skills
- TEACHING ASSISTANTSHIP:
- Linkedincom/in/digamber-kalivemula/
- Programming Languages:
- Python, R, Java
- BigData technologies: Apache Spark, Hadoop, Hive
- DataBase Management: PostgreSQL, MySQL, SparkSQL
- Cloud technologies: DataBricks, GCP, AWS
- Libraries: Py-spark, Py-arrow, Django, fastAPI, Numpy, Pandas, scikit-learn
- Others: Unix, Linux, Docker, REST API, GIT(version control)
- File Formats: Parquet, Avro
- Database Development
- Apache Spark
- Bug Fixes
- Log Files
Accomplishments
- Data engineer with strong expertise in building and optimising big data pipeline and improving data accessing speeds.Successfully implemented a data warehousing project in cloud for data migration and architectured the ETL process to accommodate data migration in large scales
- Designed and built various parts of the data warehouse to utilise available computing resources very efficiently
- I am well versed in advanced SQL practices, python development, Apache spark and design patterns for large-scale systems.
Timeline
DATA ENGINEER
314e India Corporation
01.2020 - 01.2022
SOFTWARE DEVELOPER/ETL DEVELOPER
314e India Corporation
01.2019 - 01.2020
Master's - Data Science
University of Alabama
Bachelor of Technology - Computer Science and Engineering
VNR Vignana Jyothi Institute of Engineering & Technology