General Compute
Head of Infrastructure
San Francisco · Hybrid · Director · Full-time · Posted Sep 1, 2026
- Experience
- 7+ years in infrastructure, SRE, or platform engineering, with at least some experience at an inference, ML, or HPC shop.
About the job
Own and build the infrastructure for an inference cloud, including its control plane, traffic gateway, observability, accelerator fleet operations, and incident response.
What you'll do
- Own the inference control plane, gateway and load balancer, observability, and capacity planning.
- Coordinate operational issues with the ASIC partner and bring up pre-fill on a second hardware platform.
- Build on-call and incident response practices and hire and grow the infrastructure team.
What they ask for
- 7+ years in infrastructure, SRE, or platform engineering, including some experience at an inference, ML, or HPC shop.
- Production-scale Kubernetes experience, including debugging.
- Experience building effective observability practices and handling real on-call incidents.
- Comfort owning vendor relationships and their production issues.
Nice to have
- Production experience with non-NVIDIA accelerators or model-serving stacks such as vLLM, TGI, TensorRT-LLM, or SGLang.
- Data-center-scale network fabric experience, prior infrastructure team management, or hardware-boundary experience.
Tools
Kubernetes, vLLM, TGI, TensorRT-LLM, SGLang, RoCE, InfiniBand
AI focus
Operate infrastructure for AI inference workloads, including ASIC-based decode, GPU-based pre-fill, model placement, and request routing.
Model-written summary · the original posting is authoritative