Senior Machine Learning Engineer, Inference - USA

What you'll do

• Optimize model serving and inference performance, including GPU utilization, latency, and throughput.
• Scale and maintain distributed, multi-GPU or multi-node inference systems for production workloads.
• Own the model-serving lifecycle, from containerization and optimization to reliable production operation.

What they ask for

• Deep understanding of modern inference serving frameworks and optimization techniques.
• Proficiency in C++, CUDA, Rust, or highly optimized Python, including profiling and optimizing NVIDIA GPU code.
• Experience with distributed systems and scaling, such as Kubernetes, Ray, load balancing, or multi-GPU/multi-node inference.
• PhD in CS, Physics, or Math, or equivalent practical experience building backend or ML systems.

Nice to have

• Non-trivial systems programming projects, major inference-engine open-source contributions, or deep-dive technical write-ups.
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, or speculative decoding.

Pay

$250,000 - $300,000+ USD base salary

Tools

vLLM, TRT-LLM, C++, CUDA, Rust, Python, Kubernetes, Ray

AI focus

Realtime AI inference and model serving, with emphasis on inference optimization, acceleration, and scaling.