AFROZA KHANOM
Summary
Senior QA Automation Engineer with 12+ years of experience delivering automation, API, UI, backend and cloud testing across cybersecurity and AdTech platforms. Specializing in AI Quality Engineering, LLM testing, GenAI evaluation, Playwright automation, Python, REST APIs, root cause analysis, risk analysis and test strategy. Experienced validating AI-generated outputs, prompt structures, Claude Code assisted workflows, Playwright MCP-based automation concepts, reusable Claude.md documentation, RAG evaluation, embeddings, vector databases, context windows, tokens, prompt engineering, LangChain/LangGraph concepts, and AI-assisted debugging and refactoring. Strong background collaborating with engineering teams to improve release quality through automation, security reviews and scalable QA processes.
Overview
11
11
years of professional experience
Work History
Senior QA Automation Engineer
Dune Security
New York, USA
11.2025 - Current
- AI Quality Engineering • GenAI Testing • Test Strategy • Cybersecurity
- Designed and maintained Playwright/Python Architected the organization's first scalable Playwright + Python automation framework, reducing manual regression effort by 70% while improving release confidence across UI, API, and backend services.
- Introduced risk-based testing by prioritizing critical business workflows, historical defect trends, and production incidents, enabling engineering teams to detect high-impact defects earlier in the SDLC.
- Partnered with engineering leadership to transform QA from a validation function into a quality engineering practice through automated quality gates, root cause analysis, release readiness reviews, and risk assessments.
- Established reusable AI-assisted engineering workflows using Claude Code and structured prompt libraries, improving developer productivity while maintaining code quality through systematic human validation.
- Designed evaluation strategies for AI-generated code, prompts, and test cases by assessing correctness, maintainability, business logic alignment, and security risks before production adoption.
- Developed repeatable prompt engineering guidelines (Claude.md) that standardized AI-assisted development across multiple engineers and reduced inconsistencies in generated automation.
- Performed deep defect investigations using production logs, MongoDB validation, backend services, API traces, and frontend debugging to identify systemic failures rather than isolated bugs.
- Reduced regression escape risk by implementing automated regression suites covering authentication, RBAC, reporting, user management, and security-critical workflows.
- Collaborated closely with Product, Engineering, Security, and DevOps teams to continuously improve release quality through data-driven testing strategies for cybersecurity platform features.
- Used AI-assisted development workflows with Claude Code to accelerate debugging, refactoring and test generation.
- Reviewed AI-generated code, test cases and implementation suggestions before production use.
- Created reusable Claude.md guidance and prompt templates for consistent engineering workflows.
- Built comprehensive UI/API regression suites and performed risk analysis, root cause analysis and security-focused testing.
- Collaborated with developers using AI-assisted pair programming to improve maintainability and productivity.
- Validated AI-generated outputs for correctness, reliability and business logic.
- Worked with concepts including RAG, embeddings, vector databases, context windows, tokens, prompt engineering, LangChain, LangGraph and MCP architecture to support AI-enabled testing initiatives.
- Designed reusable test strategies for AI-assisted workflows, prompt validation and evaluation suites.
Senior QA Automation Engineer
Siprocal
01.2023 - 10.2025
- Built automation frameworks using Python, Selenium, Cypress and Pytest; led QA for Spark migration; validated Kafka/MSK, S3, Athena, Druid and Aerospike pipelines; supported CI/CD; tested OpenAI-powered internal tools and vector database search capabilities.
QA Automation Engineer
Integral Ad Science
09.2017 - 12.2022
- Built Selenium/Python automation suites, API testing, backend validation, CI/CD integration, regression strategy and test framework improvements across advertising analytics platforms.
Automation Engineer
Percolate
01.2016 - 09.2017
- Developed Selenium/Python automation, frontend/backend/database testing, SQL validation and automated regression suites.
Education
B.S. - Management Information Systems
New York Institute of Technology
Caltech AI & Machine Learning Bootcamp
Skills
- AI Quality Engineering
- LLM Testing
- GenAI Evaluation
- Prompt Engineering
- Prompt Testing
- AI Output Validation
- RAG Testing
- Embeddings
- Vector Databases
- LangChain
- LangGraph
- LangSmith
- MCP Concepts
- OpenAI API
- Claude Code
- Claudemd
- Playwright
- Pytest
- Selenium
- Cypress
- Python
- REST API Testing
- MongoDB
- MySQL
- AWS
- Root Cause Analysis
- Risk Analysis
- Security Review
- Test Design
- Test Strategy
- Automation Frameworks
Affiliations
- Designed evaluation strategies for Generative AI applications using correctness, relevance, groundedness, hallucination detection, consistency, and safety metrics.
- Validated AI-generated code, prompts, automation scripts, and engineering documentation before integration into production workflows.
- Created reusable prompt templates that improved response consistency across engineering teams.
- Evaluated AI-assisted software development using structured review criteria covering maintainability, scalability, security, readability, and business correctness.
- Applied human-in-the-loop validation practices to ensure reliable AI-assisted engineering decisions.
- Contributed to AI-enabled QA practices involving prompt validation, evaluation methodologies, retrieval-augmented generation (RAG) concepts, embeddings, vector databases, LangChain, LangGraph, and MCP architectures. (Only claim direct experience where applicable.)
AI Test Strategy
- Prompt Validation
- Prompt Regression
- AI Output Validation
- Hallucination Detection
- Consistency Evaluation
- Context Window Testing
- Groundedness Validation
- RAG Evaluation
- AI Safety Testing
- Prompt Injection Testing
- Toxicity Evaluation
- Human-in-the-loop Validation
- AI Risk Assessment
Metrics-Driven Quality
- Reduced manual regression execution by 70%
- Improved automation coverage from 20% → 85%
- Reduced escaped defects by 45%
- Increased regression execution frequency from weekly to every pull request
- Reduced release validation time from 2 days to 4 hours
- Improved defect detection earlier in the SDLC
- Reduced flaky tests by 60%
- Increased API automation coverage by 80%
- Reduced production bugs through proactive risk analysis
Timeline
Senior QA Automation Engineer
Dune Security
11.2025 - Current
Senior QA Automation Engineer
Siprocal
01.2023 - 10.2025
QA Automation Engineer
Integral Ad Science
09.2017 - 12.2022
Automation Engineer
Percolate
01.2016 - 09.2017
B.S. - Management Information Systems
New York Institute of Technology
Caltech AI & Machine Learning Bootcamp