Adaption
Inference Engineer
San Francisco · Hybrid · Full-time · Posted Oct 5, 2026
- Experience
- 5+ years in ML systems, inference infrastructure, or performance engineering.
About the job
Own and optimize the inference stack to improve model-serving cost, throughput, latency, reliability, and performance as workloads and infrastructure change.
What you'll do
- Improve throughput, cost, and tail latency through cache management, batching, speculative decoding, and quantization.
- Optimize long-context workloads and route traffic across infrastructure and external providers based on cost, capacity, and performance.
- Work with serving engines and build profiling systems to measure time, memory, and compute use.
What they ask for
- 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable cost or latency improvements.
- Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
- Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
- Strong Python skills and proficiency in C++, Rust, or another systems language; GPU performance experience.
Tools
vLLM, SGLang, TensorRT-LLM, Python, C++, Rust, CUDA, NCCL
AI focus
Optimize inference for AI models, including caching, batching, quantization, decoding, GPU performance, and serving efficiency.
Model-written summary · the original posting is authoritative