Variance
Research Engineer, Evals
$180K–$300K
San Francisco · On-site · Full-time · Posted Mar 31, 2026
About the job
Builds evaluation benchmarks, datasets, tools, and feedback loops to measure and improve AI systems handling fraud, identity, and risk workflows.
What you'll do
- Build proprietary benchmarks and datasets for fraud, identity, and risk workflows.
- Design and run offline and online evaluations, define quality metrics, and analyze model failures.
- Create reusable evaluation tools and partner with research, engineering, product, and design to improve system quality.
Nice to have
- Experience training, evaluating, or improving modern ML systems; building benchmarks, datasets, evaluation pipelines, or quality systems.
- Familiarity with LLMs, agent systems, retrieval, post-training, or adjacent areas; interest in fraud, risk, trust and safety, compliance, or regulated and adversarial domains.
Pay
$180K – $300K (base salary)
Tools
LLMs
AI focus
Evaluates and improves AI models and agents for high-stakes fraud, identity, and risk decisions.
Model-written summary · the original posting is authoritative