Aaru
Research Manager, Evaluation
New York, NY · Manager · Full-time · Posted Aug 22, 2026
- Education
- A PhD is not required if you have equivalent evidence of rigorous empirical work and research leadership.
- Visa sponsorship
- Visa sponsorship is offered.
About the job
Leads Aaru’s Evaluation Research team, developing rigorous studies and reusable infrastructure to assess whether AI-generated populations, predictions, and simulations reflect real-world outcomes and support sound decisions.
What you'll do
- Lead, hire, and develop Evaluation Researchers and research engineers while directly contributing to difficult evaluations.
- Set the evaluation agenda and standards, and build reusable datasets, testing systems, and reporting infrastructure.
- Communicate rigorous findings—including negative or inconclusive results—and ensure product and external claims match the evidence.
What they ask for
- Experience leading evaluation, measurement, or empirical research in a rigorous field.
- Ability to design studies, analyze data, write code, inspect failures, and review technical work.
- Experience managing or technically leading researchers and developing their independent judgment.
- Willingness to work in person in New York City.
Nice to have
- Experience evaluating LLM agents, multi-agent systems, synthetic populations, simulations, or decision-support tools.
- Experience building evaluation platforms, experiment-tracking systems, shared research datasets, or scientific reporting tools.
AI focus
Evaluate AI populations, predictions, agent behavior, and end-to-end simulations against real-world outcomes; develop measurement methods and infrastructure to guide system improvement and support evidence-based claims.
Model-written summary · the original posting is authoritative