Arm
Principal Software Engineer, AI Inference Runtime
$263K–$355K
Seattle, Washington · Principal · Full-time · Posted Sep 9, 2026
- Experience
- 12+ years, or equivalent demonstrated impact.
About the job
Sets technical direction and develops, optimizes, and validates production AI inference runtime capabilities for distributed workloads and evolving models.
What you'll do
- Define architecture, interfaces, and roadmap for inference runtime capabilities.
- Enable new model architectures and optimize scheduling, batching, execution, memory management, kernels, and data movement.
- Build benchmarking, validation, and safe-rollout systems; collaborate across teams and mentor engineers.
What they ask for
- 12+ years of experience, or equivalent demonstrated impact, in ML systems, high-performance systems, compilers, kernel development, or production AI inference.
- Deep understanding of modern AI inference, including model execution, Attention, MoE, batching, prioritisation, and KV-cache behavior.
- Strong programming skills in C++, Rust, Python, or a comparable language, and knowledge of concurrency, parallel programming, memory resources, and data movement.
- Proven ability to profile, debug, and optimize performance across kernels, runtimes, frameworks, operating systems, and hardware.
Nice to have
- Experience developing or modifying inference schedulers, cache managers, batching systems, or disaggregated or distributed execution paths.
- Experience optimizing kernels with accelerator programming tools, assembly, or intrinsics.
Pay
$262,700-$355,400 per year (base salary)
Tools
C++, Rust, Python
AI focus
Develops production distributed AI inference runtime capabilities, enabling and optimizing modern AI models and workloads.
Model-written summary · the original posting is authoritative