Staff Software Engineer, Inference API

What you'll do

• Design and maintain inference APIs for generation, streaming, model configuration, tool use, structured outputs, and multimodal inputs.
• Integrate models and serving runtimes, and coordinate GPU prefill with Cerebras decode through routing, state transfer, and error handling.
• Improve serving performance, correctness, reliability, observability, testing, and developer experience.

What they ask for

• 5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.
• Strong programming ability in Python and Go, plus experience with C++, Rust, or a similar systems language.
• Experience building stable APIs and integrating software across service, framework, runtime, and infrastructure boundaries.
• Experience with inference APIs; Linux, containers, Kubernetes or comparable orchestration, CI/CD, and operating latency-sensitive production services.

Nice to have

• Contributions to vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, TensorRT-LLM, or another open-source ML systems project.
• Experience with API or model-validation test systems, SDKs and developer tools, multi-model inference, or disaggregated prefill/decode architectures.

Tools

Python, Go, C++, Rust, vLLM, PyTorch, Hugging Face libraries, AMD ROCm

AI focus

Build APIs and serving infrastructure for foundation-model inference, including model integration, GPU prefill and Cerebras decode, and capabilities such as streaming, tool use, structured outputs, and multimodal inputs.