Conduct quality reviews of AI-powered customer conversations and automated moderation decisions, assessing their alignment with Bumble’s community policies, operational processes, and quality standards.
Analyse policy enforcement gaps and nuanced cases, identifying where automated decisions may be inconsistent, unclear, or misaligned with policy intent and surfacing actionable insights for improvement.
Perform accurate case labelling to support AI model training, evaluation pipelines, and ongoing quality benchmarking, demonstrating Excellence through careful and consistent application of established guidance.
Review and maintain golden sets that reflect current policy standards and provide reliable ground truth for assessing AI quality, accuracy, and precision.
Report quality issues through established workflows, taking ownership of accurate categorisation, clear documentation, and actionable feedback, and seeing insights through to meaningful impact.
Collaborate purposefully with the AI Performance & Enablement Manager on scoring consistency, calibration, recurring quality issues, and opportunities to strengthen AI reliability and the member experience.
Identify patterns and share learnings that can improve AI accuracy and precision, customer satisfaction, and operational efficiency, approaching emerging challenges with Curiosity and an agile mindset.
Use QA platforms and AI-supported tools, including systems such as Cinder, Zendesk QA, and Bumble’s internal assessment tooling, to monitor quality and understand AI behaviour, reviewing outputs critically for accuracy, bias, privacy, and fairness and applying human judgment before acting on results.
About you
Requires 1+ years of experience, though we welcome candidates with alternative backgrounds that demonstrate equivalent skills.
Bring experience or transferable skills from Trust & Safety, content moderation, Customer Experience, quality assurance, or another fast-paced, policy-driven environment.
Have a sharp eye for detail and can objectively evaluate conversations, moderation decisions, and other interactions against established policies, processes, and quality standards.
Bring familiarity with conversational AI systems, chatbots, automated moderation, or AI-supported workflows, with the Curiosity to continue learning how these systems behave, where their limitations lie, and how human judgment can improve their effectiveness.
Understand, or are motivated to learn, quality assurance practices such as case labelling, calibration, consistent scoring, golden-set evaluation, and clear documentation of quality findings.
Are comfortable learning digital tools and platforms and can use data and quality insights to identify meaningful patterns while adapting your approach as systems, policies, and priorities evolve.
Collaborate purposefully with colleagues, take ownership of your deliverables, and see work through from insight to impact while welcoming feedback as an opportunity to learn and improve.
Apply AI thoughtfully and responsibly, assessing outputs for accuracy, bias, privacy, and fairness rather than relying on them without review, and demonstrate Bumble’s values of Respect, Curiosity, and Excellence in how you support our community and colleagues.
Responsibilities
What you’ll do
Conduct quality reviews of AI-powered customer conversations and automated moderation decisions, assessing their alignment with Bumble’s community policies, operational processes, and quality standards.
Analyse policy enforcement gaps and nuanced cases, identifying where automated decisions may be inconsistent, unclear, or misaligned with policy intent and surfacing actionable insights for improvement.
Perform accurate case labelling to support AI model training, evaluation pipelines, and ongoing quality benchmarking, demonstrating Excellence through careful and consistent application of established guidance.
Review and maintain golden sets that reflect current policy standards and provide reliable ground truth for assessing AI quality, accuracy, and precision.
Report quality issues through established workflows, taking ownership of accurate categorisation, clear documentation, and actionable feedback, and seeing insights through to meaningful impact.
Collaborate purposefully with the AI Performance & Enablement Manager on scoring consistency, calibration, recurring quality issues, and opportunities to strengthen AI reliability and the member experience.
Identify patterns and share learnings that can improve AI accuracy and precision, customer satisfaction, and operational efficiency, approaching emerging challenges with Curiosity and an agile mindset.
Use QA platforms and AI-supported tools, including systems such as Cinder, Zendesk QA, and Bumble’s internal assessment tooling, to monitor quality and understand AI behaviour, reviewing outputs critically for accuracy, bias, privacy, and fairness and applying human judgment before acting on results.
About you
Requires 1+ years of experience, though we welcome candidates with alternative backgrounds that demonstrate equivalent skills.
Bring experience or transferable skills from Trust & Safety, content moderation, Customer Experience, quality assurance, or another fast-paced, policy-driven environment.
Have a sharp eye for detail and can objectively evaluate conversations, moderation decisions, and other interactions against established policies, processes, and quality standards.
Bring familiarity with conversational AI systems, chatbots, automated moderation, or AI-supported workflows, with the Curiosity to continue learning how these systems behave, where their limitations lie, and how human judgment can improve their effectiveness.
Understand, or are motivated to learn, quality assurance practices such as case labelling, calibration, consistent scoring, golden-set evaluation, and clear documentation of quality findings.
Are comfortable learning digital tools and platforms and can use data and quality insights to identify meaningful patterns while adapting your approach as systems, policies, and priorities evolve.
Collaborate purposefully with colleagues, take ownership of your deliverables, and see work through from insight to impact while welcoming feedback as an opportunity to learn and improve.
Apply AI thoughtfully and responsibly, assessing outputs for accuracy, bias, privacy, and fairness rather than relying on them without review, and demonstrate Bumble’s values of Respect, Curiosity, and Excellence in how you support our community and colleagues.
Linked-in
Google