NVIDIA
Distinguished Engineer, Scaled Out Inferencing
$320K–$489K
US, CA, Santa Clara +1 · Principal · Full-time · Posted Aug 11, 2026
- Location
- US, CA, Santa Clara
US, CA, Remote - Experience
- 16+ overall technical-role years; 7-10+ years of leadership experience.
- Education
- BS/MS or higher or equivalent experience in systems/software engineering or related engineering fields.
About the job
Leads NVIDIA’s strategy and technical roadmap for large-scale AI inference, architecting distributed model-serving systems and managing their full lifecycle across cloud and enterprise environments.
What you'll do
- Architect high-throughput, low-latency distributed inference systems and pipelines for large-scale AI workloads.
- Lead hardware-software optimization, including GPU resource management and kernel- and driver-level performance tuning.
- Direct model lifecycle strategy and technical execution, including deployment, versioning, scaling, operations, and ecosystem collaboration.
What they ask for
- 16+ overall years in technical roles, with a recent long-term focus on AI infrastructure and more recent direct experience in large-scale inference orchestration.
- 7-10+ years of leadership experience.
- Proficiency in GPU architecture, hardware acceleration, low-level performance tuning (CUDA, kernels), and cloud-native architectures for multi-tenant model serving.
- Track record building secure, highly available, durable production distributed systems and delivering technically complex solutions.
Nice to have
- Real-world experience building systems for AI/ML workloads.
- Open-source ecosystem experience, including projects such as Dynamo, TensorRT-LLM, vLLM, SGLang, or Ray.
Pay
320,000 USD - 488,750 USD base salary
Tools
CUDA, Dynamo, TensorRT-LLM, vLLM, SGLang, Linux, Kubernetes, Ray
AI focus
Leads strategy and architecture for large-scale AI inference and model serving, including orchestration, optimization, deployment, versioning, and scaling.
Model-written summary · the original posting is authoritative