Research Engineer - Inference

What you'll do

• Deploy models from research checkpoints into production serving infrastructure.
• Optimize inference latency, throughput, and cost using techniques such as quantization, distillation, KV-cache optimization, batching, and custom kernels.
• Build high-performance serving systems and tooling that help researchers ship models safely and quickly.

What they ask for

• Experience deploying and serving ML models in production, ideally for latency-sensitive or real-time applications.
• Strong GPU programming and inference optimization skills, such as CUDA, Triton, TensorRT, vLLM, or SGLang.
• Ability to profile, diagnose, and remove bottlenecks across the serving stack, and build tools to measure performance.
• Demonstrated ability to solve difficult engineering problems through artifacts such as projects, designs, or GitHub contributions.

Tools

CUDA, Triton, TensorRT, vLLM, SGLang

AI focus

Deploying and optimizing frontier AI models for real-time production inference.