Lead the validation and verification of AI systems by architecting automated testing frameworks to evaluate non-deterministic outputs, including hallucination detection, consistency checks, and factual accuracy against gold standard datasets for Large Language Models (LLMs) and predictive engines.
Design and implement automated evaluation metrics such as RAGAS, faithfulness, and answer relevance to validate Retrieval-Augmented Generation (RAG) pipelines, ensuring accurate citation of internal technical documentation and regulatory texts.
Responsibilities
Key Responsibilities
Lead the validation and verification of AI systems by architecting automated testing frameworks to evaluate non-deterministic outputs, including hallucination detection, consistency checks, and factual accuracy against gold standard datasets for Large Language Models (LLMs) and predictive engines.
Design and implement automated evaluation metrics such as RAGAS, faithfulness, and answer relevance to validate Retrieval-Augmented Generation (RAG) pipelines, ensuring accurate citation of internal technical documentation and regulatory texts.
Linked-in
Google