Aaru
Researcher, Evaluation
New York, NY · Full-time · Posted Aug 22, 2026
- Education
- A PhD is not required if the candidate can demonstrate equivalent research depth and empirical rigor.
- Visa sponsorship
- Visa sponsorship offered.
About the job
Owns research and measurement that determine whether Aaru’s AI populations, predictions, and simulations are reliable enough to inform real-world decisions.
What you'll do
- Define measurable constructs and design studies to assess simulation realism, prediction quality, and decision usefulness.
- Build evaluation code and reusable infrastructure; analyze results, inspect failures, quantify uncertainty, and communicate findings.
- Collaborate across research, engineering, product, and deployment teams while maintaining independent evaluation standards.
What they ask for
- Rigorous research experience in ML evaluation, statistics, behavioral science, computational social science, economics, psychometrics, experimental design, or a related field.
- Ability to write code, analyze large datasets, and build evaluation systems.
- Experience with observational data, sampling, statistical power, uncertainty, causal threats, selection, leakage, and condition shift.
- Clear communication of technical evidence to technical and nontechnical audiences.
Nice to have
- Experience evaluating LLM agents, multi-agent systems, synthetic populations, probabilistic models, simulations, or decision-support tools.
- Experience building evaluation platforms, regression suites, experiment-tracking systems, shared datasets, or reporting tools.
AI focus
Evaluates AI agents, populations, predictive systems, and end-to-end simulations, including their behavioral fidelity, calibration, reliability, and decision impact.
Model-written summary · the original posting is authoritative