Design, build, and maintain automated CI/CD testing pipelines to validate AI agent behavior before deployment and throughout the development lifecycle.
Develop and maintain golden datasets containing regression prompts, edge cases, and synthetic scenarios to measure agent accuracy, safety, performance, and latency.
Implement fast, cost-effective, deterministic evaluation checks before introducing LLM-as-a-judge approaches, including JSON schema validation, regular expressions, API status checks, and embedding similarity.
Create comprehensive test suites for multi-turn agent conversations, evaluating context retention, state consistency, reasoning behavior, and safeguards against infinite tool-calling loops.
Design and implement adversarial testing scenarios to assess resilience against prompt injection, malformed inputs, unexpected tool responses, and other potential security vulnerabilities.
Linked-in
Google