Research Scientist, Agentic Data & Benchmarking

What you'll do

• Design reproducible evaluations and reliable evaluation harnesses for agentic capabilities; analyze results and diagnose anomalies.
• Source, generate, curate, and quality-check trajectories, tool-use data, task datasets, RL environments, and reward signals.
• Collaborate on research publications, benchmarks, and tooling, and translate capability goals into measurable data and evaluations.

What they ask for

• 2+ years of experience emphasizing evaluations and/or training-data curation for ML systems.
• Strong Python and PyTorch development experience.
• Demonstrated experience designing and investigating evaluations or curating and generating training datasets.
• Hands-on experience using LLM agents.

Nice to have

• Experience with reinforcement learning, reward design, or RL environment construction for LLMs.
• Background in statistics and experimental design.

Pay

$150,000 – $450,000/year USD base salary

Tools

Python, PyTorch, Ray

AI focus

Build and evaluate agentic language models, including agent data, tool use, RL environments, reward signals, and benchmarks.