Raydar
Research Engineer
$200K–$250K
San Francisco, California · Full-time · Posted Oct 1, 2026
- Experience
- 3 to 6 years building evaluation frameworks for AI systems whose outputs vary from run to run.
About the job
Own and develop the evaluation framework, tools, and human-review processes used to determine whether AI models and software features are ready to release.
What you'll do
- Build release-gating evaluation suites and human-rated review rubrics.
- Develop dashboards and tooling, and partner with research, product, engineering, and external colleagues to measure quality.
- Audit existing evaluations, close gaps, set the release quality bar, and help shape the research roadmap.
What they ask for
- Strong engineering skills in tooling, dashboards, instrumentation, and fast experiment loops.
- Ability to design rubrics for human reviewers.
- Track record of owning and protecting a quality bar under product pressure.
- Hands-on real-world evaluation experience, such as at a data or evaluation company or advanced AI research group.
Nice to have
- Experience evaluating agentic, on-device, or tool-using systems.
- Exposure to consumer AI, device-maker quality assurance, or early-hire/fast-growing companies.
Pay
USD 200,000 to 250,000 per year (base salary)
AI focus
Evaluate AI systems, including capabilities, behavior, regressions, and model quality; establish human and automated evaluation methods.
Model-written summary · the original posting is authoritative