Senior Quality Engineer II (AI & Data) at Salla

Makkah Al Mukarramah, Makkah Region, Saudi Arabia -

Full Time

Start Date

Immediate

Expiry Date

05 Jul, 26

Salary

0.0

Posted On

06 Apr, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Pipeline Validation, API Testing, Generative AI Evaluation, LLM-as-a-judge, Agentic AI Validation, Security Testing, Prompt Injection Defense, UI Automation, SQL, Machine Learning, Selenium, Mabl, Playwright, Root Cause Analysis, E-commerce, OWASP

Industry

Information Technology & Services

Description

Role Overview: As a Senior Quality Engineer on the AI Team, you will be responsible for defining and executing the quality strategy for our AI-driven products. You will work at the intersection of data engineering and frontend application development, ensuring that our machine learning models are reliable, our data pipelines are accurate, and the final user experience is seamless. Responsibilities: Data Pipeline Validation: Test the integrity of data ingestion and transformation processes to ensure models are trained on high-quality datasets. API & Model Testing: Conduct deep API testing using tools like APIDog to validate model responses, handling complex edge cases such as zero-value parameters or unauthorized access. Generative Model Output Accuracy: Define and implement evaluation frameworks to measure the quality of generative AI outputs — including relevance, factual accuracy, coherence, and hallucination rate — using techniques such as LLM-as-a-judge, human-in-the-loop scoring, and automated benchmark suites. AI Agent Response Validation: Design test strategies to assess the correctness and reliability of agentic AI responses, covering tool-call accuracy, multi-step reasoning integrity, goal completion rate, and graceful failure handling for out-of-scope or adversarial inputs. Security Testing: Proactively identify vulnerabilities like request interception and price manipulation to safeguard the system including prompt injection and jailbreak attempts specific to generative AI surfaces. UI Automation: Build and maintain scalable automation suites using low-code tools like mabl, aiming for high Mabl Test Coverage %. AI Tooling Advocacy: Champion the use of AI assistants like Jira Rovo for requirement analysis and test case generation to increase team velocity. Root Cause Analysis (RCA): Lead the RCA process for production bugs and developer-rejected tickets to continuously improve the "Definition of Done". Senior Experience: 8+ years of relevant work experience Data Proficiency: Strong SQL skills and experience testing data-heavy applications or ML models. Automation Mastery: Proficiency in automation tools (e.g., Selenium, mabl, Playwright) Security Mindset: Understanding of OWASP principles and practical experience in security testing (request interception, label tracking). Domain Knowledge: Experience in e-commerce and critical customer-flow analysis is highly preferred.

Responsibilities

The Senior Quality Engineer will define and execute quality strategies for AI-driven products, focusing on data pipeline integrity and model accuracy. Responsibilities include conducting deep API testing, validating generative AI outputs, and implementing automated testing suites for AI agents and UI components.