Inworld AI
Staff / Senior Staff Machine Learning Engineer, Inference - USA
$270K–$430K
Mountain View, California, USA · Hybrid · Senior Staff · Full-time · Posted Apr 7, 2026
- Experience
- No specific number of years stated; equivalent practical experience building backend or ML systems is accepted in place of a PhD.
- Education
- PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
About the job
Optimize and operate high-performance, reliable inference systems for realtime AI applications at scale.
What you'll do
- Optimize model serving for latency, throughput, and reliability, from research handoff through production deployment.
- Build and scale inference infrastructure for multi-GPU and multi-node workloads and high volumes of concurrent connections.
- Profile and accelerate systems running on NVIDIA GPUs.
What they ask for
- Deep understanding of modern inference serving frameworks and optimization techniques.
- Hands-on experience with model acceleration, such as quantization, distillation, caching, continuous batching, paged attention, or speculative decoding.
- Proficiency in C++, CUDA, Rust, or highly optimized Python, with performance profiling experience.
- Experience with distributed systems and scaling, including Kubernetes, Ray, custom load balancing, or multi-GPU/multi-node inference.
Nice to have
- Non-trivial systems programming projects, major inference-engine open-source contributions, or in-depth technical write-ups.
- PhD in CS, Physics, or Math.
Pay
$270,000 - $430,000+ USD base salary
Tools
vLLM, TRT-LLM, C++, CUDA, Rust, Python, NVIDIA GPUs, Kubernetes
AI focus
Optimize and scale inference for LLMs, speech models, and realtime AI applications.
Model-written summary · the original posting is authoritative