ByteDance
Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start
Seattle, Washington · Full-time · Posted Oct 28, 2026
- Education
- Bachelor’s or master’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field; candidates must be completing or have recently completed the degree.
About the job
Designs and operates evaluation systems and improvement feedback loops for LLM-based agents, turning agent traces and user feedback into production improvements.
What you'll do
- Design agent evaluation systems for task success, tool use, reasoning quality, and reliability; build benchmarks and automated judging pipelines.
- Analyze agent traces and user feedback to identify failure patterns and guide system improvements.
- Work with research, platform, and product teams to bring evaluation methods into production.
What they ask for
- Completing or recently completed a bachelor’s or master’s degree in computer science, AI, machine learning, data science, or a related field.
- Solid foundation in machine learning and deep learning.
- Hands-on experience with LLM-based systems through research, internships, or projects.
- Strong Python skills, experience with a mainstream ML or agent evaluation framework, and demonstrated research or engineering ability.
Nice to have
- Publications at top-tier ML/NLP venues, especially in agent learning, self-improving/self-evolving/RSI, or agent evaluation.
- Experience with evaluation methodology, LLM post-training, reasoning and planning, continual learning, feedback-driven optimization, or large-scale log and trace analysis.
Tools
Python
AI focus
LLM agent evaluation, benchmarking and automated judging; analyzing agent performance and feedback to drive capability improvements.
Model-written summary · the original posting is authoritative