Senior Product Manager, Inference Platform

What you'll do

• Set product vision and strategy for platform capabilities, including APIs, capacity and cost management, performance, and model-serving infrastructure.
• Translate user needs and infrastructure constraints into requirements and roadmaps, and partner with engineering, research, and user groups through launch.
• Own the inference product lifecycle and track developments across models, serving frameworks, competitive dynamics, and use cases.

What they ask for

• 12+ years of experience delivering complex technical products.
• Bachelor’s degree or higher, or equivalent experience.
• Deep familiarity with AI/ML systems, inference-serving frameworks, their tradeoffs, and the needs of model publishers, application developers, and cloud operators.
• Experience with inference API design, versioning, performance, hardware efficiency, and developer experience.

Nice to have

• Experience with advanced inference techniques such as disaggregated prefill/decode, KV cache management, speculative decoding, or continuous batching.
• Background in large-scale systems, developer platforms, cloud infrastructure, MLOps tooling, capacity planning, quota management, or resource scheduling.

Pay

208,000 USD - 327,750 USD base salary (Level 5); 240,000 USD - 379,500 USD base salary (Level 6)

Tools

TensorRT-LLM, vLLM, Triton Inference Server

AI focus

Product management for large-scale AI inference and model serving, including platform capabilities and inference optimization.