Arm
Staff Software Engineer, AI Inference Runtime
$209K–$283K
Seattle, Washington · Staff · Full-time · Posted Sep 9, 2026
- Experience
- 8+ years of experience, or equivalent demonstrated impact, in ML systems, high-performance systems, compilers, kernel development, or production AI inference.
About the job
Sets technical direction and develops and optimizes distributed AI inference runtime capabilities for efficient production execution of advanced models.
What you'll do
- Define architecture, interfaces, and roadmap for inference runtime capabilities.
- Enable model architectures through operator support, validation, and optimization of scheduling, execution, memory, and KV-cache efficiency.
- Profile bottlenecks, optimize kernels and data movement, and build benchmarking, validation, and rollout systems.
What they ask for
- Deep understanding of modern AI inference, including model execution, Attention, MoE, batching, prioritisation, and KV-cache behavior.
- Strong programming skills in C++, Rust, Python, or a comparable language, with knowledge of concurrency, parallel programming, memory resources, and data movement.
- Proven ability to profile, debug, and optimize performance across kernels, runtimes, frameworks, operating systems, and hardware.
Nice to have
- Experience developing or modifying inference schedulers, cache managers, batching systems, or disaggregated or distributed execution paths.
- Experience optimizing kernels with accelerator programming tools, assembly, or intrinsics.
Pay
$209,100-$282,900 per year (USD, base salary)
Tools
C++, Rust, Python
AI focus
Develops production AI inference runtime systems, including model execution, scheduling, batching, KV-cache and memory management, kernel optimization, and distributed workloads.
Model-written summary · the original posting is authoritative