ServiceNow
Senior Software Engineer, Agent Eval Platform
$161K–$274K
Mountain View, CALIFORNIA, United States · Senior · Full-time · Posted Aug 13, 2026
- Experience
- 5+ years building production backend or infrastructure systems; experience in at least 3 of the listed technical areas.
- Visa sponsorship
- For roles requiring access to controlled technology subject to export control regulations, employment is contingent on obtaining any required export license or other government approval.
About the job
Build the evaluation platform that measures multi-step AI agent behavior and uses calibrated evaluation signals to improve agent performance.
What you'll do
- Develop evaluation orchestration, including scenario execution, scheduling, retries, run isolation, and reporting.
- Build agent observability and tracing, including OpenTelemetry-based trajectory data and debugging tools.
- Create stateful simulation environments and advance use of evaluation signals to optimize agents.
What they ask for
- 5+ years building production backend or infrastructure systems.
- Strong in Python or Go (ideally both).
- Experience designing and operating systems that handle real traffic at scale.
- Experience in at least 3 of the listed areas: distributed systems, orchestration/workflow runtimes, observability internals, concurrent and async programming, data-intensive pipelines, or gRPC/protobuf design.
Pay
$161,300-274,200 USD base pay
Tools
Python, Go, Temporal, Airflow, Argo, OpenTelemetry, gRPC, protobuf
AI focus
Applied ML evaluation of AI agents, including LLM-based judging, calibration against human labels, and using evaluation signals as reward signals to optimize agents.
Model-written summary · the original posting is authoritative