HPE
Senior Inference SW Engineer
$137K–$315K
Spring, Texas, United States of America +2 · Senior · Full-time · Posted Oct 6, 2026
- Location
- Spring, Texas, United States of America
Durham, North Carolina, United States of America
Ft. Collins, Colorado, United States of America - Experience
- Minimum 8 years in Software Engineering, including 1-2+ years working directly on LLM inference runtimes or production model serving.
- Education
- Degree in Computer Science or related field.
About the job
Build and improve the LLM inference runtime and Kubernetes orchestration for HPE’s enterprise AI platform, focusing on efficient, reliable model serving on customer-owned infrastructure.
What you'll do
- Own runtime components such as engine integration, batching, KV cache management, and quantized execution; improve latency and GPU throughput.
- Develop distributed execution and orchestration capabilities, including parallelism, GPU scheduling, request routing, and autoscaling.
- Resolve customer issues, evaluate emerging inference technologies, and contribute code reviews and mentoring.
What they ask for
- Experience with LLM inference engines and ability to modify engine internals.
- Strong understanding of inference internals and tensor/pipeline parallelism, NCCL, GPU memory, and interconnects.
- Advanced Kubernetes platform architecture proficiency; strong Go and Python skills, and ability to debug and profile C++/CUDA.
- Degree in Computer Science or a related field.
Nice to have
- Upstream contributions to inference or serving projects such as vLLM, SGLang, TensorRT-LLM, llm-d, LMCache, or KServe.
- Experience with disaggregated serving, KV cache offload, RDMA technologies, GPU isolation, or on-premises and air-gapped enterprise delivery.
Pay
USD 144,000 - 273,000 in Colorado (base salary); USD 137,000 - 315,000 in North Carolina & Texas (base salary)
Tools
vLLM, SGLang, TensorRT-LLM, TGI, NVIDIA NIM, Kubernetes, Go, Python
AI focus
Develop and optimize enterprise LLM inference and serving, including runtime execution, cache management, quantization, speculative decoding, and distributed serving.
Model-written summary · the original posting is authoritative