Research Scientist, AI Evaluation Science

What you'll do

• Design experiments, advance evaluation methodology, and publish research findings.
• Partner with platform engineers to translate methods into production-ready evaluation SDKs and APIs.
• Collaborate with measurement scientists and define research priorities for evaluation science.

What they ask for

• Ph.D. in Computer Science, Machine Learning, or a closely related field, with research focus in an evaluation-adjacent area.
• Strong publication record at top-tier conferences, including first-author publications demonstrating independent contributions.
• Deep expertise in at least one specified evaluation-adjacent ML area, supported by strong mathematical foundations.
• Ability to implement complex methods from recent papers, run large-scale experiments, translate research into practical systems, and communicate effectively.

Nice to have

• Publications specifically on evaluation methodology.
• Hands-on experience with PyTorch, JAX, or TensorFlow and training or fine-tuning large language models.

Pay

$205,400 and $308,500 (base salary)

Tools

LLMs, RLHF, DPO, PyTorch, JAX, TensorFlow, SDKs, APIs

AI focus

Research evaluation methods for LLMs, agentic systems, and human-AI interaction, including preference learning, reward modeling, calibration, bias detection, active learning, and evaluator validity.