Amazon
Machine learning engineer II, AWS Inference Optimization
$165K–$224K
Santa Clara, CA · Full-time · Posted Oct 24, 2026
- Experience
- 3+ years professional software development; 2+ years system design or architecture; 1+ years software development engineer or related occupational experience; 1+ years developing large-scale software using C#, C++, Java, or Perl.
- Education
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.
About the job
Develop and productionize software and optimization techniques to improve the performance and price-performance of large language model inference on AWS services.
What you'll do
- Develop model servers, CUDA kernels, and GenAI optimization and benchmarking services.
- Implement inference optimization techniques and tune inference on Amazon SageMaker, Bedrock, and Hyperpod.
- Deliver inference software and solutions, including prototypes, in customer-facing products.
What they ask for
- 3+ years of non-internship professional software development experience.
- 2+ years of non-internship experience designing or architecting new and existing systems.
- 1+ years of software development engineer or related occupational experience, and 1+ years developing large-scale software applications, tools, systems, or services using C#, C++, Java, or Perl.
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field; knowledge of machine learning and LLM fundamentals.
Nice to have
- 3+ years of full software development life cycle experience.
- Bachelor's degree in computer science or equivalent.
Pay
165,200.00 - 223,600.00 USD annually (base salary)
Tools
Qwen, Gemma, gptoss, vLLM, SGLAng, Amazon SageMaker, Bedrock, CUDA
AI focus
Optimize large language model inference, including speculative decoding, disaggregated inference, CUDA kernel tuning, and performance benchmarking.
Model-written summary · the original posting is authoritative