Arm
Director, Product Management
$260K–$352K
San Jose, California · Director · Full-time · Posted Oct 26, 2026
About the job
Leads product strategy and execution for a next-generation AI inference platform, spanning distributed serving, compute, and control-plane capabilities.
What you'll do
- Define the vision, strategy, and roadmap for large-scale AI inference systems.
- Guide inference architecture and orchestration, including distributed serving, routing, batching, scheduling, and memory optimization.
- Partner with engineering and other teams to improve platform performance, efficiency, scalability, and customer impact.
What they ask for
- Product management experience in AI/ML infrastructure, cloud platforms, or distributed systems.
- Deep understanding of AI inference systems and large-scale serving architectures.
- Understanding of LLM inference concepts, including prefill vs decode, KV cache, and token streaming.
- Demonstrated ability to deliver scalable, high-performance platform products and collaborate directly with engineering teams.
Nice to have
- Experience with accelerator-based systems and performance optimization.
- Familiarity with TensorRT-LLM or vLLM, or experience on production-scale AI systems.
Pay
$260,000-$351,800 per year (USD, base salary)
Tools
TensorRT-LLM, vLLM
AI focus
Defines and develops infrastructure for large-scale AI inference, including serving and optimization for LLM, multimodal, and agentic workloads.
Model-written summary · the original posting is authoritative