Scale AI
Machine Learning Research Scientist, Evaluations
$166K–$207K
San Francisco, CA +3 · Full-time · Posted Aug 26, 2026
- Location
- San Francisco, CA
Seattle, WA
New York, NY
San Francisco, California, United States - Education
- Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field.
About the job
Build evaluations and benchmarks for frontier language and multimodal models, diagnose their failure modes, and inform model development through post-training expertise and research.
What you'll do
- Analyze frontier LLM and agent behavior to identify and diagnose capability, reasoning, robustness, and alignment failures.
- Design benchmarks and evaluation methods for text and multimodal model capabilities.
- Connect evaluation findings to training data and interventions, collaborate with foundation-model labs, and publish research.
Nice to have
- Master's or Ph.D. in Computer Science, Machine Learning, AI, or a related field.
- Experience with post-training techniques and LLM evaluation or benchmark development; major ML research publications and customer-facing experience are also noted.
Pay
$165,600 — $207,000 USD (base salary)
Tools
LLM, SFT, RLHF, reward modeling, preference modeling, instruction tuning
AI focus
Evaluate and diagnose frontier generative AI models, including text and multimodal systems, and use post-training methods to address failures.
Model-written summary · the original posting is authoritative