ABOUT USER INTERVIEWS

At User Interviews, we believe that the best companies in the world consistently deliver products and experiences that their customers love. We also believe that the only way to consistently build those products and experiences is to talk to your customers. Watch what they do. Understand why they do what they do. Figure out why they do things that seem irrational. And once you’ve done that once, do it again. Start having constant conversations. In short, make customers your #1 priority through user research.
That’s why we exist. We help teams set up those conversations, that research, allowing them to discover and embrace user insights. We specialize in participant recruitment and management because you cannot do good research without good participants, no matter how good your other tools may be. We work with hundreds of companies every month, including user-centric organizations like Atlassian, Amazon, and Spotify.
✨ Learn more about our culture and team here 2025 UI Retreat Video
About the Role
As part of our Strategy & Operations (S&O) team, you will be the cornerstone of task quality for AI training initiatives. You will support the development and oversight of a key program providing QA services to high-value clients, many working on cutting-edge AI models and applications. Your mission includes defining what makes a “good” written task, ensuring consistent application across a network of distributed task writers, and using technical QA to validate execution. The ideal candidate has a bias towards action and thrives in environments where they can partner with both internal stakeholders and external clients to deliver exceptional results.

BASIC QUALIFICATIONS

• 2-3 years of experience in QA, data operations, or process quality
• Strong, concise written communication; experience giving clear, constructive feedback
• Familiarity with Python and SQL
• Strong analytical and problem-solving skills, especially in ambiguous scenarios
• High attention to detail and commitment to maintaining standards at scale
• Candidates must have a United States address and be authorized to work without any Visa sponsorship

PREFERRED QUALIFICATIONS

• Experience with QA tools and workflow platforms (e.g., Jira, Confluence, Sentry, HubSpot)
• Familiarity with AI/ML training, data labeling, or annotation workflows
• Ability to translate manual QA processes into scalable or automated workflows
• Experience coordinating or managing distributed testers/task writers

• Develop and maintain quality standards across network-submitted tasks
• Provide clear, constructive feedback to drive quality and consistency
• Validate task outcomes by reviewing SQL database diffs and running/debugging Python verifier scripts
• Collaborate with stakeholders to refine task guidelines, quality protocols, and operational processes
• Manage onboarding and assessment of task writers to ensure tasker readiness—aligned with AI project requirements
• Monitor task performance metrics and writer engagement; iterate on guidelines and workflows based on insights
• Develop and document scalable processes, enablement materials, and operational plans for task writer networks
• Partner with internal stakeholders, including Revenue, Product, and Engineering, to ensure customer needs are met
• Support the Director of Strategy & Operations with customer relationship management
• Provide support across other high-value client projects, as needed