Microsoft
Sr. Software Engineer - AI Inference & Serving
$120K–$261K
Redmond, WA · On-site · Senior · Full-time · Posted Oct 9, 2026
- Experience
- 4+ years of technical engineering experience; Bachelor's degree or equivalent experience.
- Education
- Bachelor's Degree in Computer Science or related technical field, or equivalent experience.
About the job
Designs, optimizes, and scales Azure AI inference infrastructure to serve frontier AI models in production with high throughput, low latency, and high availability.
What you'll do
- Build core inference infrastructure and improve end-to-end model performance and efficiency.
- Design load scheduling and balancing strategies based on model and workload characteristics.
- Scale the platform, deliver capabilities for new generative AI models, and collaborate with internal and external partners.
What they ask for
- Bachelor's Degree in Computer Science or related technical field, or equivalent experience.
- 4+ years of technical engineering experience, including coding in C, C++, C#, or Java.
- Ability to meet Microsoft, customer and/or government security screening requirements, including the Microsoft Cloud Background Check.
Nice to have
- Experience with high-scale, reliable, low-latency, high-throughput online systems; L7 network proxies and gateways; or network architecture and protocols.
- Knowledge or experience with OSS, Docker, Kubernetes, C++, or Golang; cross-team collaboration skills.
Pay
USD $119,800 - $234,700 per year (base salary); USD $160,200 - $261,000 per year (base salary)
Tools
C, C++, C#, Java, L7 network proxies and gateways, HTTP, TCP, Docker, Kubernetes, Golang
AI focus
Hosts, optimizes, and scales inference for LLM and multimodal generative AI models, including model-serving performance, scheduling, and platform capabilities.
Model-written summary · the original posting is authoritative