NVIDIA
Senior Software Engineer - AI Inference Performance
$184K–$356K
US, CA, Santa Clara · Senior · Full-time · Posted Sep 28, 2026
- Experience
- More than 6 years.
- Education
- BS or MS in Computer Science, Computer Engineering, or a related field, or equivalent experience.
About the job
Optimize end-to-end LLM and VLM inference performance across models, serving systems, distributed runtimes, kernels, and GPU hardware, delivering measurable improvements in production deployments.
What you'll do
- Analyze, model, profile, and optimize inference workloads, latency, throughput, efficiency, and KV-cache capacity.
- Build and tune performance-critical kernels and serving techniques; establish reproducible benchmarks and regression gates.
- Collaborate across teams and contribute performance improvements to open-source inference projects.
What they ask for
- More than 6 years of full-stack LLM/VLM inference performance experience with measurable production or production-representative gains.
- Strong Python, Rust and/or C++ programming skills and hands-on CUDA or other GPU programming experience.
- Expertise in speed-of-light analysis, roofline models, microbenchmarks, NVIDIA Nsight Systems, and NVIDIA Nsight Compute.
- Deep GPU architecture knowledge and practical inference optimization experience, including distributed systems and networking.
Nice to have
- Contributions to high-performance AI projects such as TensorRT-LLM, vLLM, SGLang, PyTorch, CUDA, Triton, or NCCL.
- Experience building AI-agent-supported performance workflows, or publishing or presenting inference-performance work.
Pay
184,000 USD - 287,500 USD (base salary, Level 4); 224,000 USD - 356,500 USD (base salary, Level 5)
Tools
Python, Rust, C++, CUDA, CUTLASS, Triton, NVIDIA Nsight Systems, NVIDIA Nsight Compute
AI focus
Optimize LLM and VLM inference; optional preferred work includes AI-agent-supported profiling, bottleneck analysis, serving configuration exploration, and runtime or kernel code generation.
Model-written summary · the original posting is authoritative