Software Engineer, Evals

What you'll do

• Create and execute instruments for measuring model capabilities and behaviors.
• Manage evaluation execution infrastructure, datasets, and grading schemes.
• Build and ship reliable systems that support engineers and accelerate research.

What they ask for

• 3+ years of experience actively writing code

Nice to have

• Familiarity with AI/ML workflows, annotation tools, or evals.
• Experience creating and maintaining component libraries, design systems, or internal tools.

Pay

$175,000 - $275,000 USD (base salary)

AI focus

Develop and operate model evaluations to measure capabilities and behaviors, improve training data and experiments, track progress, and diagnose agent failures.