Member of Technical Staff - Inference

What you'll do

• Modify and extend vLLM, SGLang, and Sail’s internal inference engine.
• Profile GPU execution and design parallelism strategies for different hardware topologies.
• Write and debug GPU kernels and improve inference performance.

What they ask for

• Strong understanding of LLM mechanics, including KV cache, mixture-of-experts, and prefill versus decode.
• Great interpersonal and technical communication.

Nice to have

• Interest in MLSys research and tracking ideas such as speculative decoding and sparse attention.
• Familiarity with modern tile-based GPU programming, such as Triton, CUTLASS, or ThunderKittens, or interest in learning it.

Pay

$200K – $300K USD base salary

Tools

vLLM, SGLang, GPU, nsys, Triton, CUTLASS, ThunderKittens

AI focus

Works on LLM inference and agent hosting infrastructure, including token processing, inference engines, and GPU performance optimization.