Turing
Senior Software Engineer, AI Evaluation Infrastructure
$170K–$200K
United States · Senior · Full-time · Posted Oct 19, 2026
- Experience
- 5+ years building and operating production backend systems.
- Education
- Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
About the job
Build and operate end-to-end infrastructure for evaluating coding agents and language models, including benchmarks, large-scale evaluation runs, and task quality checks.
What you'll do
- Design and ship features across data models, APIs, background workers, and user interfaces.
- Build evaluation workflows that run and score agent trials and produce trustworthy results; improve benchmark task quality checks.
- Keep systems reliable and performant, integrate models and benchmarks, and review teammates’ code.
What they ask for
- 5+ years building and operating production backend systems, including ownership of at least one service from design through on-call.
- Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
- Strong Python and experience with Django or a similar framework, PostgreSQL, and a task queue such as Celery.
- Hands-on LLM evaluation experience beyond calling a model API, including benchmark tasks, trial scoring, and result aggregation and comparison.
Nice to have
- Experience with Harbor, Terminal-Bench, SWE-bench, Inspect, or a similar evaluation harness.
- Experience with sandboxed or containerized agents, LLM-as-a-judge or rubric-based grading, coding agents, MCP, distributed scheduling, or open-source evaluation/infrastructure projects.
Pay
$170,000 - $200,000 USD base salary
Tools
Python, Django, PostgreSQL, Celery, TypeScript, React, Docker, Kubernetes
AI focus
Build infrastructure and workflows for evaluating coding agents and language models, including benchmark trials, scoring, aggregation, task verification, and LLM-as-a-judge grading.
Model-written summary · the original posting is authoritative