Vals AI
Member of Technical Staff - Research
$130K–$185K
San Francisco · On-site · Staff · Full-time · Posted Jun 27, 2026
- Education
- Master's degree or PhD in Computer Science, NLP, Machine Learning, or related field; undergraduates with very strong research backgrounds may also be considered.
About the job
Design and build research-backed benchmarks that evaluate how well large language models perform challenging real-world tasks.
What you'll do
- Design, develop, and validate novel benchmarks; analyze model performance and communicate findings.
- Collaborate with foundation model labs, enterprises, and infrastructure teams to implement benchmark evaluations at scale.
- Publish research findings and contribute to the evaluation research community.
What they ask for
- Master's degree or PhD in Computer Science, NLP, Machine Learning, or a related field; undergraduates with very strong research backgrounds may also be considered.
- Published papers in reputable venues focused on NLP, ML evaluation, or benchmarking.
- Strong understanding of experimental design, statistical analysis, and evaluation frameworks.
- Proficiency in Python for research and experimentation.
Nice to have
- Experience in LLM evaluation or benchmarking research.
- Familiarity with foundation model architectures and capabilities.
Pay
$130K – $185K (base salary)
Tools
Python, Django, React, AWS, CDK
AI focus
Research and develop benchmarks and evaluation methods for assessing LLM and foundation-model capabilities on real-world tasks.
Model-written summary · the original posting is authoritative