Microsoft
Principal Applied Scientist for Copilot Evals
$143K–$304K
Redmond, WA · On-site · Principal · Full-time · Posted Oct 5, 2026
- Experience
- 6+ years with a bachelor’s degree, 4+ with a master’s, or 3+ with a doctorate in a listed or related field; or equivalent experience.
- Education
- Bachelor’s, master’s, or doctorate in a listed or related field; or equivalent experience.
About the job
Sets the scientific direction for customer-grounded Copilot quality by turning enterprise workflows and customer feedback into evaluations, learning signals, and model improvement priorities.
What you'll do
- Define quality strategy and evaluate whether Copilot evaluation sets represent customer needs.
- Translate customer research, product signals, and incidents into reusable evaluation, regression, and post-training assets.
- Develop evaluation methods, graders, and metrics, and diagnose quality losses across Copilot systems.
What they ask for
- Bachelor’s degree in a listed or related field and 6+ years of related experience; or master’s and 4+ years; or doctorate and 3+ years; or equivalent experience.
- Degree fields include Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or a related field.
Nice to have
- Experience designing evaluations, metrics, experiments, datasets, graders, or reward functions for AI or agentic systems.
- Experience with large language models, copilots, agents, tool use, retrieval-augmented generation, or enterprise grounding.
Pay
USD $142,800 - $274,800 per year (base salary); USD $188,000 - $304,200 per year (base salary)
AI focus
Customer-grounded Copilot evaluation and quality improvement, including AI and agent behavior assessment, reward signals, and post-training priorities.
Model-written summary · the original posting is authoritative