Databricks
Staff Software Engineer, Agent Quality
$200K–$265K
New York City, New York +1 · Staff · Full-time · Posted Sep 24, 2026
- Location
- New York City, New York
San Francisco, California - Experience
- 6+ years industry experience building software systems
About the job
Builds the infrastructure and workflows to evaluate Databricks’ AI agents and use evaluation results to improve agent quality throughout development, training, and production.
What you'll do
- Establish evaluation infrastructure for Genie Agents, including benchmarking, regression detection, and quality measurement.
- Connect evaluation results and production signals to agent training and iterative improvement.
- Set technical direction for agent quality infrastructure and collaborate across research and product teams.
What they ask for
- Strong Python programming skills and experience building production or research infrastructure.
- Experience building or operating distributed systems, data pipelines, or large-scale infrastructure.
- Ability to design reliable systems that produce trustworthy, reproducible signals.
- Ability to work across research and product teams, with strong ownership and technical judgment.
Nice to have
- Experience with devtools, CI/CD platforms, testing frameworks, observability tooling, or benchmarking infrastructure.
- Familiarity with measuring LLM or agent quality through evaluations, experimentation platforms, or production monitoring.
Pay
$200,000 — $265,000 USD (salary range)
Tools
Python
AI focus
Evaluate and continuously improve AI agents, including Genie Agents, by feeding evaluation and production signals into agent development and training.
Model-written summary · the original posting is authoritative