Tech Lead - AI Inference

What you'll do

• Own inference infrastructure and guide design, implementation, and delivery of reliable, high-throughput, low-latency systems.
• Contribute hands-on across the serving stack and tackle inference challenges such as disaggregated prefill/decode, KV caching, RDMA transport, and GPU memory hierarchies.
• Mentor and develop engineers, and evaluate emerging inference tools to guide adoption decisions.

What they ask for

• 8+ years of professional software engineering experience, with proven experience leading engineers and owning complex production systems.
• Hands-on expertise in LLM serving, including KV cache reuse, disaggregated prefill/decode, continuous batching, and multi-tier GPU memory hierarchies; familiarity with vLLM, LMCache, NIXL/NVIDIA Dynamo, or similar.
• Strong Python and C++ skills, with knowledge of CUDA, GPU memory management, and high-performance I/O including GDS, RDMA, and NVMe data paths.
• Experience deploying and scaling GPU workloads on Kubernetes.

Nice to have

• Experience in AI/ML infrastructure or high-performance computing.
• Rust skills; familiarity with RDMA networking, bare-metal GPU clusters (H100/A100), and high-throughput distributed storage.

Tools

Python, C++, CUDA, vLLM, LMCache, NIXL/Dynamo, Kubernetes, SGLang

AI focus

Leads development of LLM inference and serving infrastructure, including KV caching, disaggregated prefill/decode, and GPU memory optimization.