Sierra
Software Engineer, Inference
$230K–$390K
San Francisco, CA · On-site · Full-time · Posted Oct 5, 2026
About the job
Build and operate Sierra’s inference systems, routing model workloads across self-hosted infrastructure and external providers while optimizing performance, reliability, and cost.
What you'll do
- Design inference architecture and serving or proxy layers across models, infrastructure, and providers.
- Build systems for routing, failover, capacity and quota management, and operate self-hosted inference on GPU infrastructure.
- Optimize inference performance with the Applied Research team and contribute to post-training infrastructure.
What they ask for
- Strong distributed systems fundamentals and deep systems thinking.
- Experience designing, building, and operating large-scale production systems.
- Sound judgment on latency, reliability, capacity, and cost tradeoffs.
- Experience owning complex infrastructure from architecture through production operation.
Nice to have
- Experience with ML infrastructure, MLOps, production inference, or serving large models at scale.
- Experience with self-hosted inference or GPU infrastructure; familiarity with vLLM or SGLang; or post-training and inference optimization experience.
Pay
$230K – $390K (base salary)
Tools
vLLM, SGLang
AI focus
Develop infrastructure for model inference, including self-hosted and provider-based serving, inference-performance optimization, and support for post-training.
Model-written summary · the original posting is authoritative