Ambral
Member of Technical Staff (Evals & Post-Training)
$165K–$325K
New York +1 · On-site · Staff · Full-time · Posted Sep 1, 2026
- Location
- New York
San Francisco - Experience
- 1–7 years building production software or machine-learning systems.
About the job
Builds replayable enterprise environments and evaluation systems, then trains and improves agents using real historical workflows and outcomes.
What you'll do
- Build systems that convert enterprise data and task definitions into runnable environments, evaluation sets, and verifiable reward graders.
- Train and evaluate agents, improving performance and inference efficiency through model, tool, context, and policy choices.
- Develop replay, observability, and scalable infrastructure for agent training and evaluation runs.
What they ask for
- Strong software skills and ability to build systems that process large, messy datasets at scale.
- Understand how environment and reward design, context, tooling, and policy behavior interact.
- Can turn ambiguous business objectives into reliably evaluable tasks and signals.
- Can move between research questions and production implementation.
Nice to have
- Experience with reinforcement-learning environments, LLM post-training, evaluation infrastructure, agent harnesses, or related systems.
Pay
$165K – $325K (base salary)
Tools
reinforcement learning, LLM post-training
AI focus
Develops evaluation environments and methods, trains and evaluates agents, and improves model performance and cost through post-training and agent-system design.
Model-written summary · the original posting is authoritative