Senior Manager of Meta Evaluation & Quality Assurance at Apple

Seattle, Washington, United States -

Full Time

Start Date

Immediate

Expiry Date

27 Apr, 26

Salary

0.0

Posted On

27 Jan, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Science, Machine Learning, Evaluation Science, Meta Evaluation, AI/ML Measurement, Statistics, Quality Assurance, Human-in-the-Loop Systems, Active Learning Pipelines, Statistical Analysis, Governance Gates, Certification Processes, Communication, Organizational Skills, Analytical Skills

Industry

Computers and Electronics Manufacturing

Description

Apple Services Engineering (ASE) powers many AI and LLM features across App Store, Music, Video and more. As these systems increasingly rely on human-in-the-loop evaluation systems, the quality of our decisions is constrained by the quality of our evaluation systems. We believe that to build exceptional LLMs, you need exceptional mechanisms to validate the signals used to train and evaluate them. DESCRIPTION As the Senior Manager of Meta Evaluation & Quality Assurance, you will lead a specialized team of Data Scientists and Machine Learning Engineers who evaluate the evaluators. You will move beyond basic validation to lead the strategy and technical development of ML-based validation frameworks and automated data quality validation pipelines. You will set strategy, guide execution, and work cross functionally to deliver a cohesive quality system that combines machine learning with human-in-the-loop processes to ensure our metrics are trustworthy, robust, and decision-ready. MINIMUM QUALIFICATIONS 8+ years of experience in Data Science, Machine Learning, or Evaluation Science, with 3+ years leading technical teams Strong background in Meta Evaluation, AI/ML measurement, statistics, or quality assurance methodologies. Demonstrated success in designing Human-in-the-Loop (HITL) machine learning systems or active learning pipelines. Masters degree in Statistics, Data Science, Machine Learning or related field. PREFERRED QUALIFICATIONS PhD in Statistics, Computer Science, Machine Learning, or related field Deep understanding of evaluation pipelines, calibration techniques, and statistical process control Experience building ML models specifically designed for quality estimation, anomaly detection, or disagreement modeling Proficiency in Python or R for statistical analysis and reasoning about evaluation data Experience defining governance gates or certification processes for AI systems Proven ability to manage complex methodological and technical programs in dynamic, fast-paced environments. Exceptional communication, organizational, and analytical skill.

Responsibilities

Lead a specialized team of Data Scientists and Machine Learning Engineers to evaluate the evaluators. Develop ML-based validation frameworks and automated data quality validation pipelines.