Cohere
Staff Software Engineer, Inference Infrastructure
$190K–$325K
San Francisco +3 · Hybrid · Staff · Full-time · Posted Jan 12, 2026
- Location
- San Francisco
New York
Toronto
Montreal - Experience
- 5+ years of engineering experience running production infrastructure at a large scale.
About the job
Build and operate Cohere’s model-serving platform, deploying optimized language models through API endpoints in low-latency, high-throughput, highly available environments.
What you'll do
- Develop, deploy, and operate infrastructure for serving Cohere’s large language models.
- Deploy optimized NLP models to production and support reliable, scalable inference.
- Work with teams and customers on customized deployments and operational needs.
What they ask for
- Experience designing highly available distributed systems with Kubernetes and GPU workloads.
- Experience supporting Kubernetes development and production, and troubleshooting complex Linux environments.
- Experience with cloud or hybrid infrastructure, and compute, storage, network resource, and cost management.
- Experience with Golang, C++, or another language designed for high-performance scalable servers; understanding of distributed systems and accelerator characteristics.
Pay
$190K – $325K (base salary; currency not stated)
Tools
Kubernetes, GCP, Azure, AWS, OCI, Linux, Golang, C++
AI focus
Deploy and operate optimized NLP models and large language model inference systems; account for accelerator characteristics affecting inference latency and throughput.
Model-written summary · the original posting is authoritative