Aurora
Senior Software Engineer, ML Evaluation & Introspection Infrastructure
$146K–$260K
Seattle, WA +2 · Remote · Senior · Full-time · Posted Oct 19, 2026
- Location
- Seattle, WA
Mountain View, CA
Pittsburgh, PA - Experience
- Typically 5+ years of professional software engineering experience building backend, data, or ML infrastructure.
About the job
Build and operate infrastructure for evaluating, searching, and inspecting self-driving models using real-world driving data, while developing AI-assisted analysis tools.
What you'll do
- Build evaluation pipelines and reports to compare model versions, track metrics, and detect regressions.
- Own embedding generation and vector search over driving data, and build model introspection tools.
- Develop web and CLI tools for engineers and AI agents, including workflows using LLMs, VLMs, and agents to analyze driving scenarios and evaluation results.
What they ask for
- Strong Python and proficiency with SQL.
- Designed and operated large-scale batch or data pipelines on cloud infrastructure, including backfills, retries, and cost management.
- Built systems for measurement or evaluation, such as metrics, experimentation, evaluation, or analytics platforms.
- Experience with distributed compute and container orchestration, such as Kubernetes or AWS Batch.
Nice to have
- Experience with ML evaluation infrastructure, embeddings and vector search, or model introspection and interpretability tooling.
- Experience using LLMs, VLMs, or agentic workflows for analysis, labeling, or data curation.
Pay
$162,000 - $260,000 per year (base salary; Mountain View, CA and Seattle, WA); $146,000 - $234,000 per year (base salary; Pittsburgh, PA)
Tools
Python, SQL, Kubernetes, AWS Batch, LLMs, VLMs
AI focus
Apply LLMs, VLMs, and agents to triage, label, cluster, and summarize driving scenarios and evaluation results; build tools designed for AI agents to use.
Model-written summary · the original posting is authoritative