Amazon
Senior Machine Learning Engineer, SageMaker Inference
$168K–$227K
Bellevue, WA · Senior · Full-time · Posted Oct 28, 2026
- Experience
- 5+ years for each of the three stated basic-qualification experience areas; the posting specifies no alternatives.
About the job
Build and optimize SageMaker’s infrastructure for deploying and serving generative AI models at scale, focusing on inference performance.
What you'll do
- Design kernel and inference-pipeline optimizations to reduce latency and increase throughput for LLM serving.
- Profile, benchmark, and tune inference performance across hardware and software stacks.
- Collaborate on production inference solutions, review code, mentor engineers, and evaluate optimization tools and techniques.
What they ask for
- 5+ years of non-internship professional software development experience.
- 5+ years of programming experience with at least one software programming language.
- 5+ years leading design or architecture of new and existing systems, including design patterns, reliability, and scaling.
- Experience as a mentor, tech lead, or leading an engineering team.
Nice to have
- 5+ years of full software development life cycle experience.
- Experience with CUDA or ML/low-level kernels, or LLM inference and serving optimization.
Pay
168,100.00 - 227,400.00 USD annually (base salary)
Tools
SageMaker, CUDA, LLM, CPU, GPU, Inferentia, AWS
AI focus
Generative AI inference serving, including LLM performance optimization, kernel tuning, profiling, benchmarking, and model execution techniques such as disaggregated serving and KV caching.
Model-written summary · the original posting is authoritative