Director Digital Product - AI Quality Assurance and Testing at UnitedHealth Group Canada

, , -

Full Time

Start Date

Immediate

Expiry Date

30 Apr, 26

Salary

0.0

Posted On

30 Jan, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

AI Quality Assurance, Testing Strategy, Data Integrity, Machine Learning, Generative AI, AI Ethics, Incident Management, Team Leadership, Collaboration, Risk Management, Automation, Documentation, Data Engineering, AI Frameworks, Postmortems, Employee Engagement

Industry

Hospitals and Health Care

Description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. The Director, AI Quality Assurance & Testing is the first of its kind and critical role for accelerating the AI agenda across UHC Consumer Operations. This position will report to the VP of AI Delivery and Business Integration. This leader’s mission is to drive 3 things: Build or select AI/ML testing tool (model testing, synthetic data, monitory) for Consumer Product and where needed for consumption across UHG Partner with and provide 24/7 support to US QA lead for AI Program Pre-Production Testing & AI Program Postproduction Testing/Monitoring Own incident management, including but not limited to root cause analysis, roll backs or retraining decisions This leader’s mission is to lead a testing strategy team of approximately 20 individuals (and closely partner a US counterpart who is primarily responsible for testing delivery) to rigorously test seamless AI enabled consumer experiences, maximize AI agentic delivery and ensure UHC wide ambitious cost savings due to consumer self-service adoption. Primary Responsibilities: Define the AI testing strategy vision, standards and KPI’s Automate regression tests across testing models and versions Build RLAIF frameworks (Reinforcement learning AI feedback loop) leveraging AI-assisted evaluation and reinforcement learning for continuous feedback loops at scale Establish program KPI’s and regularly report process to executive leadership, including periodic model re-validation and retraining readiness Build and maintain documentation for model behavior, testing evidence and decision logs- goal is to see a reduction in AI related production incidents Research and provide recommendations on industry regulations, internal risk frameworks for AI testing- expectation of re platforming AI frameworks minimally annually Deliver on pre-defined audit pass rates for AI testing, including but not limited to preparing AI systems for internal and external audits Ensure that CP level Testing SLA’s for AI powered testing are met by AI testing delivery team (US counterpart) Maintain risk assessments and mitigation documentations Build edge case testing framework Develop post launch QA playbook covering live shadow testing against ground truth datasets, A/B and multivariant testing to drive adoption of AI risk management practices Implement automated data testing (schema validation, drift detection, labeling audits) Partner with data engineering to enforce and establish data and quality control Champion responsible AI practices ensuring fairness, transparency and accountability in AI Models Hire, mentor and scale a high performing AI delivery and business integration organization Drive a culture of blameless postmortems to strength resilience and accountability Act as champion for AI career growth ensuring pathways are clear, equitable and aligned to business needs Support key priorities of increasing employee and management engagement by promoting Our United Culture and DEI concepts and activities with leaders and within the organization Engage with internal and external partners alike to position the company as a trusted AI innovator Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so Required Qualifications: Undergraduate degree or equivalent experience 12+ years in testing strategy and/or operations with 2+ years of AI/ML focused testing leadership roles Experience in establishing data integrity and governance frameworks at scale Solid understanding of machine learning, generative AI, data platforms and AI ethics Track record of delivering AI products to market with measurable impact Proven track record of: Working collaboratively in a highly matrixed organization Building relationships with people from diverse areas of an organization Bringing fresh ideas and new thinking to the team (with views that change opportunistically) Being energized and passionate with the ability to persuasively sell ideas and concepts Operating effectively at executive levels Thinking broadly about overall business issues with both an “inside-out” and “outside-in” perspective Managing, motivating and developing a team and attracting and retaining outstanding talent Empowering team members and holding them accountable Rapidly assessing situations and absorbing & applying new information to decision-making and direction setting Demonstrated intellectual capacity and sound judgment to lead in changing and ambiguous environments Demonstrated confidence and perseverance in addressing challenges and overcoming obstacles History of addressing difficult issues with candor, speaking the truth, expressing a point of view and taking a stand Preferred Qualifications: Experience partnering with AI forward companies like Amazon, Google, Microsoft Familiarity with global data protection and AI regulatory frameworks Well-rounded in AI literacy and upskilling programs At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone–of every race, gender, sexuality, age, location and income–deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes — an enterprise priority reflected in our mission.

How To Apply:

Incase you would like to apply to this job directly from the source, please click here

Responsibilities

The Director will define the AI testing strategy vision, establish program KPIs, and lead a testing strategy team to ensure rigorous testing of AI-enabled consumer experiences. They will also manage incident responses and drive a culture of accountability and continuous improvement.