Santhosh Kumar Soma

• Highly motivated and results-oriented Gen AI Testing Expert proficient in testing both Gen AI solutions and Gen AI-enabled applications
• Experienced in designing, implementing, and leading LLM and AI solution testing strategies tailored for Gen AI technologies
• Understanding of NLP, Omni-Channel testing, TTS, and STS as they apply to Gen AI platforms and applications
• Expertise in prompt engineering, data acquisition, LLM migration strategies, and intelligent test automation methodologies specific to Gen AI environments
• Pioneered Smart Automation Test Framework customized for Gen AI testing requirements
• Established Benchmarking Test Framework for consistent quality and performance evaluation of Gen AI solutions and applications
• Led End-to-End Test Automation initiatives tailored for Gen AI-enabled apps, covering complete automation cycles
• Advanced Prompt Engineering skills applied to automate complex test scenarios inherent to Gen AI technologies
• Utilized tools like PyRIT for single and multi-turn injection attacks to ensure robust security measures in Gen AI applications
• Successfully led LLM Migration Validation during transitions from different Gen AI models, ensuring seamless integration and performance continuity
• Developed Intelligent Test Automation libraries specifically designed for Gen AI applications to enhance performance and user experience
• Identified and addressed gaps in test plans specific to Gen AI testing, implementing new strategies for comprehensive evaluation
• Prioritized high-risk scenarios through Risk-Based Testing Strategies, optimizing resource allocation for Gen AI testing projects
• Proficient in leveraging platforms like Hugging Face for data acquisition and management tailored for Gen AI testing needs.
• Leverage LLMs for multilingual test data generation, ensuring comprehensive coverage of language-specific features and functionalities.
• Evaluate Aisera Domain Specific Models performance using LLMs, assessing accuracy and fluency in understanding and generating various supported Domains(IT, HR, Banking, Insurance..etc)

• Managed a team of 10+ QA professionals, ensuring efficient daily testing and release activities.
• Streamlined partner relationships for Mobile Threat Defense (MTD) solutions, saving Ivanti's QA capacity by 45 man-days.
• Rebuilt and stabilized a 60% portion of the Offshore QE Team within 6 months.
• Increased customer satisfaction by achieving a 70% reduction in customer defects (FY 2022).
• Championed continuous quality improvement by analyzing quality gaps and defining action plans.
• Oversaw defect tracking, reporting, and resolution to identify recurring issues and enhance future releases.
• Achieved 90% release accuracy by consistently delivering iOS apps on schedule.
• Achieved significant automation , 100% Post Production Sanity and 78% of Regression testing.
• Instrumental in raising overall Unit and Integration Test coverage, improving overall Code coverage by 28%.
• Oversaw defect Tracking , reporting and resolution process using jira to identify repeated issues and improve release going forward.

San Jose, California - January 2021 - March 2021

• Designed and built In-house Stress Test Framework "Pulsar" for MobileIron applications (iOS & Android)
• Developed Core Services in Pulsar framework for App Crashes, Battery Utilization, and CPU Utilization
• Automated End-to-End Use Cases with Pulsar Framework for Longevity and Stress Testing on both platforms
• Analyzed Longevity Test Results, reported issues, and performed root cause analysis
• Documented E2E Test Case development and debugging with Pulsar Test Framework

MobileIron

Mountain View, California - January 2020 - December 2020

• Demonstrated expertise in Appium internals, customizing and fixing issues for MobileIron needs (WebDriverAgent, AppiumforMac, WinAppDriver, UIAutomator2)
• Identified and resolved issues in Appium components to support early beta releases for MobileIron App certification (Android & iOS)
• Designed and implemented a common Quality Engineering Executive Dashboard
• Automated End-to-End Cases for Pre Apple App Submission
• Extended Automation Framework to leverage existing tests for branded Client application certification
• Developed iOS Test Application "Open URL in Safari" to expedite execution and stabilize tests
• Automated complex E2E features across Mobile Device Management, Mobile App Management, Mobile Threat Defense Solutions, AFW (Android for Work), DEP (Device Enrollment Program), various MDM configurations (VPN, WIFI, Exchange), and FIDO Zero Trust passwordless authentication

Hyderabad, India - July 2017 - January 2020

• Led automation efforts for 10+ Android and iOS apps (Mobile@Work, Go Client, Docs@Work, Web@Work, PIV-D, etc.)
• Increased automation coverage and resolved challenges with device automation (Android & iOS) (collaboration with MobileIron Client Team leaders)
• Extended Next Gen Framework to support latest GA and Beta releases from Android, iOS, and Mac platforms
• Added and extended features to Next Gen Framework to meet MobileIron needs
• Designed and implemented features in Next Gen Framework:
• Retry with failure recovery (handling unknown issues in the In-House Device Farm)
• Test distribution (parallel execution of E2E use cases on multiple devices)
• Scaled In-House Device Lab from ~10 devices to +50+ devices (Android, iOS, Mac) within 12 months
• Added new drivers in Next Gen Automation for latest OS platforms (Mac OS, Watch OS)
• Certified MobileIron applications on latest devices from various OEMs (LG, Samsung, Sony, Google, Apple, OnePlus, Zebra, etc.)
• Managed and upgraded In-House Device Lab infrastructure for Device Automation
• Developed Proof of Concepts with Next Gen framework for MobileIron initiatives (Core to Cloud migration, Post-Production verification, Battery Consumption Tests, On-Demand Automation, etc.)
• Evaluated Code Coverage for all MobileIron Applications with E2E Automation
• Mentored and coached Client Teams on automating manual use cases and upskilling automation capabilities
• Actively participated and drove organization-level automation initiatives
• Automated nearly 70% of End-to-End functional test cases on Android and iOS platforms
• Upgraded existing drivers in Next Gen with latest changes from underlying automation tools
• Managed backend infrastructure and added new features to the On-Demand Automation Portal
• Actively engaged with the Appium community to identify and fix issues related to supporting Beta and GA releases from Android and iOS platforms

• Sorted out daily test automation failures in the builds and integrated test results with Test Rail.
• Automated UI Tests with Sahi OS and API Tests with SoapUI Pro for Idera SqlSafe and SQLDM.
• Handled and Managed Test Complete Licenses for Windows Automation.
• Part of Idera Client Acceptance Team, reviewed and approved the Other Vendor Automation Tests,maintained VM Infra for effective Utilization.
• Assisted team in overcoming various technical and non-technical obstacles or limitations.

• Responsible for identifying and addressing daily test automation failures and deployment problems.
• Performed Code Coverage for each Release using Clover to ensure correctness of application and to identify untested areas of application.

• Developed UI automation frameworks using Web Driver, Java, and TestNG to automate UI tests.
• Implemented headless browser automation for onboarding cloud tenants in production environments.
• Integrated API Rest Client with Rest Assured in Java for UI automation to interact with backend services.
• Created custom reports using XML and XSLT with TestNG Listeners to enhance test reporting capabilities.

• Developed framework libraries for Snapfish E-commerce Application Teams(CORE, I18N, PS, MOBILE and Sanity verticals)
• Migrated QTP Scripts with Selenium RC + Java automation.
• Configured 12 Automation Servers at San Francisco with Selenium Grid and Hudson(Jenkins) to make sure that the tests are run in parallel on different systems and different environments.