SambaNova
Senior ML Infrastructure Engineer
$200K–$275K
Austin, Texas, United States +3 · Senior · Full-time · Posted Jun 4, 2026
- Location
- Austin, Texas, United States
San Jose, California, United States
Austin, Texas, United States
San Jose, California, United States - Experience
- 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving.
- Education
- B.S. in Computer Science, Electrical Engineering, or related field
About the job
Designs and operates production-grade LLM inference infrastructure and APIs for SambaNova’s RDU serving stack, taking inference techniques from prototype to production.
What you'll do
- Productionize inference techniques on RDU, including decoding, caching, and long-context inference, to improve performance and cost.
- Integrate vLLM and adjacent serving frameworks with RDU, and own public inference APIs for SambaStack and SambaCloud.
- Build accuracy verification and regression infrastructure; collaborate across teams to ship inference features.
What they ask for
- Bachelor’s degree in Computer Science, Electrical Engineering, or a related field.
- 5+ years building and operating large-scale distributed systems, ideally in ML serving.
- Experience designing and maintaining production services with strict latency, throughput, and availability requirements.
- Proficiency in Python; working knowledge of modern LLM inference techniques and familiarity with serving stacks such as vLLM, TensorRT-LLM, or SGLang.
Pay
$200,000 — $275,000 USD base salary
Tools
RDU, vLLM, TensorRT-LLM, SGLang, Python, SambaStack, SambaCloud
AI focus
Develops and productionizes LLM inference features, including speculative and constrained decoding, function/tool calling, prompt caching, and long-context inference.
Model-written summary · the original posting is authoritative