LLM Inference Systems Engineer

What you'll do

• Design and operate disaggregated serving, including prefill/decode pools, KV-cache transfer, routing, batching, and failure recovery.
• Optimize and benchmark multi-node inference; profile performance and improve latency, throughput, and cost.
• Productionize serving on Kubernetes with reliability, observability, autoscaling, and safe rollouts.

What they ask for

• Production experience with distributed systems, HPC systems, or large-scale ML inference platforms.
• Strong understanding of LLM inference, including parallelism, continuous batching, KV-cache management, and prefill/decode behavior.
• Hands-on GPU communication and networking experience; strong Python and working proficiency in C++ or another systems language.
• Experience profiling and debugging across system layers, deploying production workloads on Kubernetes, and a Master's or PhD.

Nice to have

• Direct experience with disaggregated serving, KV-cache transfer, NVIDIA Dynamo/NIXL, SGLang, vLLM, or TensorRT-LLM.
• Experience with GPU performance or networking technologies, inference features, open-source projects, or benchmarking new GPU platforms.

Tools

Python, C++, Kubernetes, NCCL, NVLink, NVSwitch, InfiniBand, RoCEv2

AI focus

Build and optimize distributed serving systems for large language models, including prefill/decode disaggregation, KV-cache transfer, and multi-node inference.