Human Data Annotator at AGI, Inc.

San Francisco, California, United States -

Full Time

Start Date

Immediate

Expiry Date

03 Jul, 26

Salary

0.0

Posted On

04 Apr, 26

Experience

0 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Yes

Skills

Data annotation, Machine learning, Agent-based trajectories, Reinforcement learning, Model alignment, Attention to detail, Written communication, Data quality assurance, Pattern recognition, UI evaluation

Industry

technology;Information and Internet

Description

Think Different. Build the Future. 🚀 Our Mission Build everyday AGI. Trustworthy, consumer-grade agents that redefine human–AI collaboration for millions. Software shouldn’t wait for commands; it should partner with you, amplifying what you can do every single day. Why AGI, Inc. We’re a stealth team of elite founders and AI researchers, with backgrounds spanning Stanford, OpenAI, and DeepMind. We’re industry leaders in mobile and computer-use agents, bringing these capabilities to consumer scale. Grounded in years of agent research, our AI is designed with trustworthiness and reliability as core pillars, not afterthoughts. We are supported by tier-1 investors who funded the first generation of AI giants; now they’re backing us to build the next: everyday AGI. (Watch the demo) If you see possibility where others see limits, read on. About the Role We're looking for experienced data annotators to help train and align our universal AI agents. You'll be evaluating agent behavior across computer and mobile interfaces, providing the nuanced feedback that shapes how our models learn and improve. This is not a mechanical task. Your judgment defines what "helpful," "safe," and "aligned" mean in practice. You'll work directly with ML researchers to refine guidelines, surface failure patterns, and build the datasets that drive our next breakthrough. You'll focus on quality, consistency, and insight, ensuring every annotation moves us closer to agents people can trust. What You'll Do Evaluate agent trajectories: Review and annotate agent behavior across desktop, web, and mobile environments — labeling actions, identifying failure modes, and assessing task completion quality. Judge decision quality: Evaluate agent decision-making at each step: Was this the right action? Was it efficient? Did it align with user intent? Provide structured feedback: Deliver qualitative feedback on agent behavior, including edge cases, reasoning errors, and UI misinterpretations. Collaborate with researchers: Work directly with ML researchers to refine annotation guidelines, surface patterns in model failures, and inform training priorities. Build benchmark datasets: Contribute to high-quality datasets that drive measurable performance improvements. Minimum Qualifications 1+ years of experience annotating agent-based trajectories, reinforcement learning data, or similar sequential decision-making tasks Strong understanding of what constitutes model quality — you can distinguish between a correct action, a suboptimal action, and a hallucinated one Familiarity with model alignment concepts: helpfulness, harmlessness, honesty, and how annotation choices influence model behavior Track record of working alongside researchers — you're comfortable with ambiguity, can propose annotation schema improvements, and understand how your work feeds into training pipelines Excellent attention to detail and ability to maintain consistency across high volumes of data Clear written communication skills for documenting edge cases and providing actionable feedback Why This Role Matters Models learn from data. Data quality determines model quality. Your annotations are the ground truth. You will directly shape how our agents behave — what they prioritize, how they reason, and whether users trust them. The patterns you identify and the feedback you provide will inform the next generation of training runs. Our Culture 🏢 All in, in person — work moves faster face-to-face 🚀 Ship by default — speed and polish can coexist 🤝 One band, one sound — radical candor, zero politics Perks 🏥 Competitive company-sponsored medical, dental, and vision insurance ✈️ Top-tier relocation and immigration support How to Apply Send us: A link — or 60-second video — of something you built and why it mattered Your resume or LinkedIn Two sentences on the hardest challenge you've solved Every exceptional candidate hears back within 48 hours. If you see possibility where others see limits, we'd love to meet you.

Responsibilities

Evaluate and annotate agent behavior across desktop, web, and mobile interfaces to improve model performance. Collaborate with ML researchers to refine annotation guidelines and identify failure modes in agent decision-making.