OpenRelay
Member of Technical Staff, Inference
San Francisco, CA, US +1 · Staff · Full-time · Posted Oct 4, 2026
- Location
- San Francisco, CA, US
Seattle, WA, US - Experience
- 3+ years of industry experience, with production LLM serving a real part of it.
- Education
- No degree required; equivalent experience counts.
About the job
Own and improve OpenRelay’s production serving layer for open-weight models on GPUs, including performance, reliability, and request routing.
What you'll do
- Optimize serving performance, cost, and reliability across models and GPUs.
- Bring new open-weight models into production and tune serving configurations.
- Design request routing and scheduling; benchmark, troubleshoot, and contribute general fixes upstream.
What they ask for
- 3+ years of industry experience, including production LLM serving.
- Production experience with vLLM, SGLang, and NVIDIA Dynamo.
- Strong Python and PyTorch, with ability to read and modify large codebases.
- Demonstrated measurable improvements in throughput, latency, or cost, supported by defensible numbers.
Nice to have
- Contributions to an open-source inference engine; GPU kernel or low-level performance work (CUDA, ROCm, or Triton).
- Experience with mixture-of-experts models, multi-node inference, or an early-stage startup.
Pay
competitive salary and equity
Tools
vLLM, SGLang, NVIDIA Dynamo, Python, PyTorch, CUDA, ROCm, Triton
AI focus
Serve and optimize open-weight models for production LLM inference on GPUs.
Model-written summary · the original posting is authoritative