Staff Software Engineer, AI Inference

What you'll do

• Build and operate scalable, cost-effective LLM serving systems; optimize performance across latency, throughput, GPU utilization, and memory efficiency.
• Set the technical roadmap and architecture for inference infrastructure, model serving, runtime optimization, and deployment operations.
• Evaluate and integrate inference technologies, partner with ML engineers to productionize models, and establish benchmarking practices.

What they ask for

• Significant experience designing and operating production AI inference systems, including building or leading production LLM serving infrastructure.
• Deep experience with one or more modern inference runtimes or frameworks, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Hugging Face TGI, or NVIDIA Dynamo.
• Strong background in distributed systems, backend infrastructure, or high-performance platform engineering; experience operating GPU infrastructure in production and optimizing GPU inference.
• Proficiency in Python and at least one systems programming language (Go, Rust, or C++); proven technical architecture leadership and strong communication skills.

Pay

$190- $230K USD base salary; $190,000 — $230,000 USD base salary

Tools

Python, Go, Rust, C++, vLLM, SGLang, TensorRT-LLM, Triton Inference Server

AI focus

Builds and optimizes production inference infrastructure for large language models, including model serving, runtime optimization, deployment, routing, autoscaling, scheduling, and monitoring.