Member of Technical Staff, AI Evaluation

What you'll do

• Study customer and engineer use of Archie, identify engineering successes and failures, and create actionable evaluations.
• Recreate real-world engineering tasks and failure modes for repeatable testing.
• Develop and improve automated judges and human evaluation methods, checking judge alignment with expert judgment.

What they ask for

• Technical background in a physical engineering domain such as power systems, power electronics, mechanical engineering, or aerospace.
• Strong understanding of agentic AI systems sufficient to design and reason about evaluations.
• Experience creating AI evaluation tasks and judges and comparing automated evaluation with human judgment.
• Ability to work directly with customers, engineers, subject matter experts, partners, and internal technical teams.

Pay

$200K – $300K (base salary)

Tools

Archie

AI focus

Evaluates an AI engineer agent for physical engineering, including building evaluation tasks, automated judges, and human evaluation processes.