Member of Technical Staff, Infrastructure

What you'll do

• Develop and operate distributed training and model-serving systems, including batching, caching, routing, and autoscaling.
• Reduce cost per token through quantization, speculative decoding, kernel optimization, and scheduling improvements; profile and resolve performance bottlenecks.
• Set and meet latency and reliability targets, and collaborate with training teams on models that are efficient to serve.

What they ask for

• Experience training or serving LLMs at scale using PyTorch, vLLM, SGLang, or a system you built.
• Strong systems fundamentals in concurrency, memory, networking, and performance measurement; working knowledge of GPU programming (CUDA or Triton) and transformer training and inference internals.
• Fluency in Python and a systems language such as C++, Rust, or Go, plus hands-on experience running GPU workloads on Kubernetes.
• Comfort owning production systems and responding when they break.

Nice to have

• Contributions to an open-source inference engine or kernel library; experience with multi-tenant serving, LoRA serving, or model multiplexing.
• Experience at an early-stage startup or as a founder.

Tools

Kubernetes, PyTorch, vLLM, SGLang, CUDA, Triton, Python, C++

AI focus

Build infrastructure for training and serving custom AI models, including inference optimization and collaboration on model architectures and formats that are cheaper to serve.