RUPERT SINGH HUMPHRIES
Torrance,USA
Summary
Experienced QA Engineer with over a decade of experience spanning streaming platforms, mobile apps, and AI systems. Proven ability to evaluate large language models (LLMs), create test data, and lead prompt evaluation workflows. Currently, I am expanding my skills in automation and ML tooling to contribute more deeply to AI-driven products. Strong in fast-paced environments, cross-team collaboration, and quality-first execution.
Overview
17
17
years of professional experience
Work History
AI Quality Engineer
ServiceNow
Los Angeles, CA
03.2024 - Current
- Specialize in manual and auto Topic Execution evaluation under Kristen Cooley and Edward Cheung
- Lead Topic Execution under my manager, while also trained to support GenAI, QnA, and conversational AI workflows through iterative evaluation, prompt testing, and scoring
- Review and score Auto Evaluation Level 1 & 2 outputs, flagging issues around hallucination, relevance, and coherence
- Assist in synthetic dataset creation to improve model coverage and catch edge cases
- Known for delivering under tight timelines, adapting to new tools quickly, and mentoring others on evaluation quality
- Trained on broader LLM systems and currently expanding Java/Python skills for future automation
Quality Assurance Analyst
NBA
New York, NY
10.2022 - 11.2023
- Led QA across web, mobile, OTT, and connected TV platforms for NBA, WNBA, and G League apps
- Created and executed test plans and automation scripts; logged and managed defects using JIRA
- Performed regression testing and exploratory testing to identify performance and functional issues
- Built dashboards and collaborated with cross-functional teams to meet release goals and improve quality metrics
Streaming Ops Test Engineer
Pluto TV
West Hollywood, CA
10.2021 - 10.2022
- Oversaw QA across global OTT platforms; validated content across U.S., LATAM, and EMEA markets
- Conducted live-stream and VOD playback validation, CC, DRM, audio/video sync, and deep link testing
- Used Charles Proxy and Wireshark to debug live channel issues and verify ad placement and server communication
- Improved channel coverage and user experience across 15+ supported platforms including mobile and smart TVs
Quality Assurance Analyst
Xandr / AT&T
El Segundo, CA
06.2015 - 05.2021
- Managed QA for ad delivery, live/VOD streaming, and set-top-box integrations across DIRECTV and AT&T platforms
- Certified new ad formats including Prebid and Prefetch integrations
- Conducted manual and automated testing on iOS, Android, tvOS, Roku, FireTV, and web platforms
- Validated beacons, impressions, network tracking, and video performance using tools like Charles and Wireshark
- Supported QA for multiple streaming apps including DIRECTV Stream, WatchTV, and AT&T TV
Technical Product Manager
AT&T
El Segundo, CA
11.2018 - 12.2019
- Led content and service development and mobile development projects with Agile/Scrum teams
- Coordinated sprint planning, OKRs, and KPIs to measure project success and resource efficiency
- Delivered dashboards and project reporting using Power BI for executive stakeholders
- Collaborated with product owners to groom backlogs and define user stories
Software Test Engineer
DIRECTV / AT&T
El Segundo, CA
06.2015 - 11.2018
- Manually tested streaming apps and set-top boxes including Android, iOS, Roku, FireTV, and Smart TVs
- Verified playback quality, closed captions, FCC compliance, Dolby Audio, and 4K compatibility
- Created and maintained test plans and automated test scripts using Selenium and Wireshark
- Logged issues, reproduced bugs, and performed regression testing during sprint cycles
QA Tester
Treyarch / Activision Blizzard / 2K Games
Santa Monica / Northridge, CA
01.2008 - 01.2014
- Tested AAA titles including Call of Duty: MW2, MW3, Black Ops III, and Bioshock Infinite
- Logged and tracked bugs using JIRA, DevTrack, Bugzilla, and SpiraTest
- Performed compatibility testing on PC, Xbox, PlayStation, and mobile devices
- Coordinated daily build verification sweeps and milestone test coverage
- Worked directly with dev teams to resolve critical issues and provide test case feedback
Education
Google Cloud Generative AI Crash Course -
Java & Python for QA Automation -
Skills
- Python (beginner), Java (beginner), Selenium, JUnit (learning)
- Test Case Design, Prompt Evaluation, LLM Quality Scoring
- Manual & Auto Evaluation (Levels 1 & 2) for LLMs
- Dataset Creation, Prompt Engineering, Topic Execution QA
- JIRA, Confluence, Jenkins, GitHub, Charles Proxy, Postman
- Agile/Scrum, API Testing, LLM Safety & Performance QA
Timeline
AI Quality Engineer
ServiceNow
03.2024 - Current
Quality Assurance Analyst
NBA
10.2022 - 11.2023
Streaming Ops Test Engineer
Pluto TV
10.2021 - 10.2022
Technical Product Manager
AT&T
11.2018 - 12.2019
Quality Assurance Analyst
Xandr / AT&T
06.2015 - 05.2021
Software Test Engineer
DIRECTV / AT&T
06.2015 - 11.2018
QA Tester
Treyarch / Activision Blizzard / 2K Games
01.2008 - 01.2014
Google Cloud Generative AI Crash Course -
Java & Python for QA Automation -
Similar Profiles
Benedict Johnson JBenedict Johnson J
Software Engineering Manager at ServiceNowSoftware Engineering Manager at ServiceNow
Sebastian ChavarriaSebastian Chavarria
Sales Operations Specialist at ServiceNowSales Operations Specialist at ServiceNow
Joel SamsJoel Sams
Technical Support Engineer at ServiceNowTechnical Support Engineer at ServiceNow
ADITYA SINGHADITYA SINGH
Software Engineer (IC2) at ServiceNowSoftware Engineer (IC2) at ServiceNow
TSZ HIN YANTSZ HIN YAN
Elementary Students Teacher Assistant at Supreme Learning CenterElementary Students Teacher Assistant at Supreme Learning Center