Senior Product Manager – AI Inference Performance

What you'll do

• Set the inference performance roadmap across model representation, memory, scheduling, and token generation; build reusable platform capabilities.
• Define performance strategy for agentic workloads and coordinate optimization delivery across frameworks and open-source communities.
• Own benchmark methodology, release readiness, quality and regression tracking, and customer feedback for the product.

What they ask for

• 12+ years in technology product management, or comparable time as a founder, engineering lead, or technical product owner.
• Hands-on AI/LLM inference performance optimization experience and deep familiarity with KV caching and reuse, quantization, speculative decoding, and disaggregated serving.
• Familiarity with TensorRT-LLM, vLLM, SGLang, NVIDIA Dynamo, and the serving and scheduling ecosystem.
• BS, MS, or PhD in Computer Science, Computer Engineering, or another relevant area of study (or equivalent experience).

Nice to have

• Engineering experience in LLM inference performance, such as profiling or kernel-level or serving-level optimization.
• Open-source contributions or product leadership in vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or Dynamo; production experience at scale also counts.

Pay

208,000 USD - 327,750 USD base salary

Tools

TensorRT-LLM, vLLM, SGLang, NVIDIA Dynamo, Triton Inference Server

AI focus

Product strategy and performance optimization for AI/LLM inference, including agentic and multi-turn workloads, frameworks, and benchmarking.