ByteDance
Agent Evaluation & Evolution Researcher Graduate (AML-Ark-US) - 2027 Start (PhD)
Seattle, Washington · Full-time · Posted Oct 28, 2026
- Education
- PhD in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field; completing or recently completed.
About the job
Designs evaluation systems and benchmarks for LLM-based agents, analyzes agent failures, and helps turn evaluation findings into production improvements.
What you'll do
- Design agent evaluation systems for task success, tool use, reasoning quality, and reliability.
- Build benchmarks and automated judging pipelines using rule-based checks, model-based judging, and human review.
- Analyze execution traces and user feedback to identify failure patterns and support production improvements.
What they ask for
- Completing or recently completed a PhD in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
- Solid foundation in machine learning and deep learning.
- Hands-on experience with LLM-based systems through research, internships, or projects.
- Strong Python skills and experience with a mainstream ML or agent evaluation framework.
Nice to have
- Publications at top-tier ML/NLP venues, especially in agent learning, self-improving/self-evolving/RSI, or agent evaluation.
- Experience with evaluation methodology, LLM post-training, reasoning and planning, continual learning, feedback-driven optimization, or large-scale log and trace analysis.
Tools
Python
AI focus
Evaluation and improvement of LLM-based agents, including benchmarks, automated judging, trace analysis, and feedback-driven production improvements.
Model-written summary · the original posting is authoritative