Raydar
Member of Technical Staff, Head of Quality
$180K–$280K
San Francisco, California · Staff · Full-time · Posted Sep 19, 2026
- Experience
- 1 to 5 years of relevant LLM evaluation, QA engineering, or ML data-quality experience.
- Visa sponsorship
- Visa transfers are supported.
About the job
Own quality for AI training environments and datasets through hands-on review, automated evaluation, delivery acceptance, and team leadership.
What you'll do
- Review human data and environment outputs; define quality metrics and acceptance criteria.
- Build automated evaluations, graders, and QA agents; identify reward hacking, broken verifiers, and distribution mismatches.
- Own customer-project delivery acceptance and hire, onboard, and manage quality staff.
What they ask for
- Experience building automated quality systems for ML data, agents, or large-scale pipelines.
- Technical understanding of how training data and environments support learning.
- Hands-on experience with LLM evaluations, annotation, or human-in-the-loop review.
- Ability to build graders, judge models, telemetry, and dashboards; strong attention to detail and willingness to review data manually.
Nice to have
- RL environment, verifier, or reward-design experience.
- Founder or zero-to-one product-building experience; a STEM degree.
Pay
USD 180,000 to 280,000 per year (base salary)
Tools
LLM, ML
AI focus
Build and evaluate training environments and datasets, including automated graders and QA agents, to support AI model learning.
Model-written summary · the original posting is authoritative