NVIDIA
Principal Software Engineer – Large-Scale LLM Memory and Storage Systems
$272K–$431K
US, CA, Santa Clara +2 · Principal · Full-time · Posted Jan 9, 2026
- Location
- US, CA, Santa Clara
US, WA, Remote
US, MA, Remote - Experience
- 15+ years.
- Education
- Masters or PhD or equivalent experience
About the job
Defines the technical vision and builds memory and storage systems for large-scale LLM inference across GPUs, host memory, local storage, and remote tiers.
What you'll do
- Design a unified multi-tier memory layer and integrate it with LLM serving engines for KV-cache offload, reuse, and sharing.
- Develop interfaces and protocols for disaggregated inference and low-latency KV-cache access across heterogeneous clusters.
- Set technical direction, mentor engineers, and collaborate with GPU, networking, and platform teams.
What they ask for
- 15+ years building large-scale distributed systems, high-performance storage, or ML systems infrastructure in C/C++ and Python, with production-service delivery.
- Deep understanding of memory hierarchies and experience designing multi-tier systems.
- Experience with distributed caching or key-value systems, networked I/O, and RDMA/NVMe-oF/NVLink-style technologies.
- Strong system profiling and optimization skills across CPU, GPU, memory, and network; excellent communication and cross-functional leadership skills.
Nice to have
- Open-source LLM serving or systems contributions focused on KV-cache optimization, compression, streaming, or reuse.
- Experience designing unified memory or storage layers; publications or patents in related systems areas.
Pay
272,000 USD - 431,250 USD (base salary)
Tools
C/C++, Python, vLLM, SGLang, TensorRT-LLM, GPUDirect, RDMA, NVLink
AI focus
Builds infrastructure for large-scale LLM inference, including multi-tier memory and KV-cache management; NVIDIA also states it uses AI tools in recruiting.
Model-written summary · the original posting is authoritative