Ollama
Software Engineer, Cloud
Palo Alto · On-site · Full-time · Posted Jul 9, 2026
About the job
Build Ollama’s scalable cloud inference platform for serving open models, including distributed infrastructure used by its customers.
What you'll do
- Build and scale the inference platform serving requests from ollama.com.
- Design routing and capacity systems to place workloads across GPUs and regions for cost, latency, and availability.
- Own multi-tenant infrastructure, reliability, observability, and cost controls, including isolation, quotas, metering, billing, and customer tiering.
What they ask for
- Deep experience with high-throughput, low-latency distributed systems.
- Comfort with cost/performance tradeoffs at scale and ownership of a production service end-to-end.
- Experience with Kubernetes, GPU scheduling, or inference infrastructure.
- Focus on reliability, SLOs, and capacity planning.
Nice to have
- Experience building an inference platform or managing a GPU fleet.
- Experience with billing or metering for an AI service.
Tools
Kubernetes
AI focus
Build and operate inference infrastructure for open models, including inference serving and GPU workload routing and capacity management.
Model-written summary · the original posting is authoritative