Research Engineer - Data and Evaluation

What you'll do

• Design coding and long-horizon agent evaluations; build executable environments, scoring logic, and verifiers.
• Investigate model failures and reward hacks by analyzing model outputs, harness code, and evaluation measures.
• Build RL data pipelines and collaborate with researchers to test performance changes on held-out tasks.

What they ask for

• Strong Python software engineering skills and experience building data pipelines, evaluation systems, or ML workflows used by others.
• Hands-on experience with language models or agents and understanding of how training data and evaluation choices affect behavior.
• Hands-on experience building technical evaluations or agent environments, including debugging model traces, harnesses, and scoring code.
• Ability to analyze complex data and outputs, identify failure modes, and own technical projects through experimentation, implementation, and validation.

Nice to have

• Experience with RL or post-training data, including task difficulty selection and rollout or reward distribution analysis.
• Contributions to Verifiers, Prime-RL, or comparable frameworks; work on model-based grading, long-horizon coding agents, or reward exploitation.

Pay

$150-350k (cash compensation range, including equity incentives; currency not stated)

Tools

Python, Verifiers, Prime-RL

AI focus

Build RL training data, environments, and evaluations for coding and long-horizon agents; analyze model behavior, failures, and reward exploitation.