OpenAI
Software Engineer, Model Runtime
$266K–$445K
San Francisco · Hybrid · Full-time · Posted Aug 24, 2026
- Visa sponsorship
- Candidates may need to meet certain legal status requirements under U.S. export control laws and regulations.
About the job
Build and optimize the production LLM inference runtime that executes frontier models on OpenAI custom silicon.
What you'll do
- Design runtime features including scheduling, continuous batching, memory and KV-cache management, and execution orchestration.
- Develop distributed execution across chips, hosts, and racks, and optimize latency, throughput, memory efficiency, and hardware utilization.
- Partner across systems and hardware teams to improve performance, support model and hardware capabilities, and build profiling and benchmarking tools.
What they ask for
- Strong systems programming experience in C++, Rust, Python, or comparable performance-oriented environments.
- Experience building or optimizing runtimes, distributed systems, compilers, kernels, model-serving infrastructure, or adjacent systems software.
- Understanding of modern LLM inference, including prefill and decode, batching, KV-cache tradeoffs, and model parallelism.
- Ability to profile and debug performance across multiple layers of a hardware-software stack.
Pay
$266K – $445K USD base salary
Tools
C++, Rust, Python, vLLM, SGLang
AI focus
Develop an inference runtime for frontier models, enabling efficient execution on custom AI silicon.
Model-written summary · the original posting is authoritative