ServiceNow
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
$201K–$352K
Santa Clara, California, United States · Manager · Full-time · Posted Jul 13, 2026
- Experience
- 8+ years of professional software engineering or machine learning experience, including 3+ years managing or technically leading high-performing AI/ML teams.
- Education
- Bachelor’s or higher degree in Computer Science, Data Science, Machine Learning, or a highly quantitative field.
About the job
Leads a team building evaluation infrastructure, benchmarks, and validation frameworks to measure the quality, safety, and reliability of ServiceNow’s agentic AI workflows.
What you'll do
- Design and scale automated evaluation systems, including testing harnesses and production drift monitoring.
- Develop enterprise benchmarks for agent orchestration, planning, routing, memory, and RAG pipelines; benchmark models for capability, latency, and cost.
- Recruit and mentor an AI evaluation team and partner across product, platform, and engineering teams to drive improvements.
What they ask for
- 8+ years of professional software engineering or machine learning experience, including 3+ years managing or technically leading AI/ML teams.
- Experience with frontier AI SDKs and deploying or testing agentic or probabilistic software architectures.
- Experience implementing enterprise-scale AI evaluation metrics and methods.
- Proficiency in Python; familiarity with SQL, Pandas, NumPy, and MLOps tracking platforms.
Nice to have
- Experience with knowledge infrastructure, SaaS platform architectures, or relational datasets such as Knowledge Graphs or CMDBs.
- Master’s or Ph.D. degree (a plus).
Pay
$201,300 - $352,300 USD base salary
Tools
Python, SQL, Pandas, NumPy, ROUGE, BLEU, G-Eval, RAG
AI focus
Leads agentic and generative AI benchmarking and evaluation, including assessing multi-step orchestration, tool use, enterprise-grounded reasoning, RAG, and frontier model performance.
Model-written summary · the original posting is authoritative