Runpod
Senior ML Systems Engineer, Inference
$150K–$220K
Remote - USA · Remote · Senior · Full-time · Posted Oct 1, 2026
- Experience
- 5+ years of professional systems engineering experience.
- Visa sponsorship
- Welcomes qualified candidates eligible to work in the United States; unable to sponsor employment visas.
About the job
Lead end-to-end performance measurement and optimization of production LLM inference across models, hardware, and workloads.
What you'll do
- Define inference performance metrics and build rigorous, repeatable benchmarking tools.
- Diagnose serving-stack bottlenecks and improve efficiency across single-node and multi-node GPU deployments.
- Implement production-ready runtimes, configurations, defaults, and serving-engine fixes.
What they ask for
- 5+ years of professional systems engineering experience.
- Hands-on production or serious benchmark-scale experience with vLLM, SGLang, or a comparable serving engine.
- Strong Python software engineering skills and familiarity with large, performance-critical codebases.
- Understanding of LLM inference performance and optimization techniques, with benchmarking, performance analysis, and GPU profiling experience.
Nice to have
- Experience writing or tuning GPU kernels in CUDA or Triton.
- Contributions to inference or ML systems projects; experience with multi-node GPU systems and high-speed networking is also preferred.
Pay
$150,000 - $220,000 USD base salary
Tools
vLLM, SGLang, Python, CUDA, Triton
AI focus
Optimize LLM inference speed and cost through benchmarking, profiling, serving-stack improvements, and production deployment.
Model-written summary · the original posting is authoritative