Dedicated and detail-oriented Data Engineer with experience in manipulating, preparing, and analyzing data, with 5 years of IT experience. Developed strong technical skills and a deep understanding of data processing and ETL (Extract, Transform, Load) processes. Possess a solid understanding of database concepts, and the ability to optimize data workflows.
Overview
6
6
years of professional experience
Work History
Senior Data Engineer
Early Warning
06.2024 - Current
Designed and automated AWS infrastructure using Terraform, employing reusable modules, remote state management, and infrastructure lifecycle policies to ensure robust, scalable, and secure environments tailored to service requirements, resulting in reduction of deployment time.
Designed and automated AWS infrastructure using Terraform, employing reusable modules, remote state management, and infrastructure lifecycle policies to ensure robust, scalable, and secure environments tailored to service requirements, resulting in reduction of deployment time.
Implemented a real-time Change Data Capture (CDC) pipeline to stream data from Amazon DynamoDB to downstream systems using Amazon Kinesis Data Streams and Kinesis Firehose.
Designed and implemented a MESH-based data integration framework for real-time processing of financial transactions, enabling seamless communication.
Engineered and optimized DynamoDB tables with advanced schema design, provisioned throughput capacity planning, and adaptive capacity settings to support high-velocity read/write operations required by PAZE transactional systems, achieving high availability.
Configured DynamoDB Streams to capture and process insert, update, and delete events, ensuring real-time data synchronization across multiple services.
Implemented stringent data security and compliance measures by configuring fine-grained IAM policies, enabling server-side encryption, and deploying VPC endpoints to secure data in transit and at rest for PAZE applications, ensuring compliance with industry standards.
Scala Data Engineer
ICF – CMS
09.2023 - 06.2024
Utilized AWS Glue to automate the extraction, transformation, and loading of healthcare data, optimizing data pipelines for scalability and efficiency.
Leveraged Redshift's columnar storage and parallel processing capabilities to optimize query performance and facilitate real-time data analysis for clinical decision support.
Implemented AWS Glue crawlers to automatically discover and catalog metadata from various data stores, streamlining the data preparation process for analytics and reporting.
Engineered state machines to orchestrate complex ETL workflows, ensuring efficient error handling.
Provisioned and configured AWS EMR clusters for running large-scale Spark jobs and data transformations.
Integrated CodeBuild with other AWS services (ECR, S3, Lambda) for deployment and artifact management.
Developed and maintained Jenkins pipelines for building, testing, and deploying data applications using Scala, Spark.
Software Development Engineer
Amazon services LLC
Chicago, USA
08.2022 - 09.2023
Developed scalable and resilient distributed systems using AWS services such as EC2, S3, and Lambda, ensuring high availability and fault tolerance.
Implemented data governance and security measures on Amazon Redshift, including encryption, access control, and auditing, to ensure compliance with regulatory standards and protect sensitive data.
Implemented data security and privacy measures in compliance with industry regulations by leveraging AWS services such as AWS Key Management Service (KMS), AWS Identity and Access Management (IAM), and AWS Data Encryption.
Demonstrated the value of Amazon Redshift's columnar storage and parallel processing capabilities in accelerating data analytics and driving actionable insights for business decision-making.
Familiarity with automating DynamoDB provisioning and configuration using AWS CloudFormation or Terraform, enabling infrastructure as code practices for consistency and repeatability.
Developed and maintained CI/CD pipelines using tools like Jenkins, AWS CodePipeline, or GitLab CI/CD, enabling automated build, testing, and deployment of software applications.
Programmer Analyst
Cognizant Technology Solutions
Bangalore, India
07.2019 - 11.2020
Collaborated with stakeholders to define data requirements and create custom data reports and dashboards.
Collaborated with cross-functional teams to gather requirements, analyze business needs, and translate them into SQL solutions for banking products and services.
Designed and optimized SQL queries and stored procedures to extract and manipulate financial data from databases for reporting and analysis.
Worked with robust and scalable software solutions using Java, contributing to the company's product suite.
Developed and maintained data warehousing solutions for storing and analyzing historical banking data, including customer transactions, account balances, and financial metrics.
Participated in database performance tuning and optimization efforts to enhance the speed and efficiency of banking applications and reporting tools
Developed and maintained database schemas, tables, and indexes to support banking applications and transaction processing systems
Managed and maintained the RTGS, NEFT, swift and bricks transactions in remote server