Engineering Manager, Private Cloud Compute Inference Routing

What you'll do

• Lead recruitment, development, and management of engineers and engineering managers responsible for inference routing and request scheduling.
• Set technical strategy and oversee adaptive routing, gateway capabilities, capacity management, and reliability practices.
• Build roadmaps and coordinate delivery across inference, infrastructure, security, Siri AI, Apple Intelligence, and SRE teams.

What they ask for

• Extensive software engineering experience leading teams that build and operate large-scale distributed systems in production.
• Experience delivering API gateways, request routing, load balancing, scheduling, or traffic-management systems at scale; expertise in LLM inference serving.
• Record of shipping highly reliable systems, including incident management, on-call, and reliability practices.
• Server development background in Java, C/C++, Go, or Swift; ability to drive cross-team alignment; bachelor’s degree in an engineering discipline or equivalent professional experience.

Nice to have

• Leadership developing LLM inference serving or routing platforms; experience with inference gateways and their core behaviors.
• Experience with capacity-aware routing and accelerator utilization, privacy-preserving design, observability, or Kubernetes.

Pay

$225,600 and $381,600 (base salary)

Tools

Java, C/C++, Go, Swift, Kubernetes, llm-d, NVIDIA Dynamo, vLLM

AI focus

Lead routing and scheduling for LLM inference, including cache-aware placement, KV-cache placement and reuse, and prefill/decode disaggregation, within PCC’s privacy architecture.