Microsoft
Senior Data Scientist for Copilot Evals
$120K–$261K
Redmond, WA · On-site · Senior · Full-time · Posted Oct 27, 2026
- Experience
- 1+ year with a doctorate, 3+ years with a master's degree, or 5+ years with a bachelor's degree in a listed or related field; equivalent experience is also accepted.
- Education
- Doctorate, master's degree, or bachelor's degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or a related field; or equivalent experience.
About the job
Builds measurement, evaluation, and prioritization systems that help determine where to invest in Copilot quality and whether it is improving for customers.
What you'll do
- Combine customer, usage, product, and evaluation signals into prioritization frameworks, taxonomies, quality gates, and recurring scorecards.
- Analyze evaluation coverage, quality trends, loss patterns, and offline-to-online agreement; turn findings into product, model, data, and evaluation recommendations.
- Develop analytical methods and automated pipelines for customer signals, and communicate evidence-based recommendations to partners and senior leaders.
What they ask for
- Doctorate in a listed or related field and 1+ year of data-science experience; or a master's degree in a listed or related field and 3+ years; or a bachelor's degree in a listed or related field and 5+ years; or equivalent experience.
- Experience managing structured and unstructured data, applying statistical techniques, and reporting results.
Nice to have
- Experience shaping product strategy with data in complex, high-scale environments; expertise in SQL and Python or R.
- Experience with AI product quality or LLM/agent evaluation, representative datasets, quality scorecards, customer-feedback analysis, or connecting offline evaluation to online outcomes.
Pay
USD $119,800 - $234,700 per year (base salary); USD $160,200 - $261,000 per year (base salary)
Tools
SQL, Python, R, CPU, GPU
AI focus
Measures and improves Copilot quality through customer-grounded evaluation, representative datasets, quality gates, and analysis connecting offline evaluation with online customer outcomes.
Model-written summary · the original posting is authoritative