ABOUT US:

Hippocratic AI is building safety-focused large language model (LLM) for the healthcare industry. Our team comprised of ex-researchers from Microsoft, Meta, Nvidia, Apple, Stanford, John Hopkins and HuggingFace are reinventing the next generation of foundation model training and alignment to create AI-powered conversational agents for real time patient-AI interactions.
As Senior Operations Manager, Evaluations, you will be the engine behind our clinical-safety evaluation platform—the cornerstone that proves our AI agents are safe, effective, and ready for real-world care. Thousands of licensed clinicians have used our app to score conversations, flag safety issues, and suggest improvements. Internal researchers rely on the resulting data to iterate fast.
Your mission: keep the app humming, the clinicians happy, and the research teams unblocked—all while driving down turnaround time and cost per evaluation. You will partner with Product, Engineering, Finance, and Clinical Safety to improve work flows and build processes to make evaluation of our AI agents as effective and seamless as possible.

REQUIRED SKILLS & QUALIFICATIONS

• 5 + years in operations, program management, or business process ownership—ideally in health-tech, clinical trials, digital health, or AI/ML.
• Demonstrated success scaling high-volume, high-accuracy processes (millions of transactions or thousands of clinical cases per month).
• Metrics obsession—comfortable defining KPIs, running A/B or DOE experiments, and presenting insights to executives.
• Outstanding written & verbal communication; able to translate clinical or technical nuance into clear SOPs and exec summaries.

• Run the shop: Own day-to-day operations for all clinical, safety, and user-experience evaluations—scheduling, SLAs, escalation paths, and reporting.
• Build & scale teams: Recruit, and onboard a global bench of licensed nurses and doctors; design performance-management loops that keep quality high
• Automate everything: Drive adoption of auto-evaluation tooling; partner with ML engineers to expand test coverage, add new rubrics, and tune pass-/fail thresholds.
• Process improvement: Map current workflows, identify bottlenecks, and ship iterative fixes—reducing turnaround time, cost per eval, and defect rate quarter over quarter.
• Data & dashboards: Define KPIs (e.g., evaluation cycle time, false-negative/positive rates, reviewer utilization); build real-time dashboards and monthly business reviews.
• Compliance & audit readiness: Ensure every evaluation artifact (prompts, ratings, nurse notes) is stored, traceable, and audit-ready for regulators and enterprise customers.