Amazon
Software Engineer II - AI/ML, Neuron Inference, Inference
$165K–$224K
Cupertino, CA · Full-time · Posted Oct 13, 2026
- Experience
- 3+ years of non-internship professional software development experience; 3+ years of non-internship design or architecture experience; 1+ years of experience developing large-scale software applications.
- Education
- Bachelor’s degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.
About the job
Develop and optimize distributed inference support and model performance in the AWS Neuron SDK for machine-learning workloads running on AWS Trainium accelerators.
What you'll do
- Build and tune distributed inference solutions, high-performance kernels, and infrastructure for onboarding diverse models.
- Profile and optimize performance across Neuron hardware; implement, test, and deploy system and model improvements.
- Work with customers and cross-functional engineering teams to enable models and improve inference performance.
What they ask for
- Knowledge of Python and/or C++ programming.
- Experience debugging and profiling large-scale systems, with understanding of performance, memory management, and parallel computing.
- Bachelor’s degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.
- Experience designing and developing large-scale software applications using C#, C++, Java, or Perl.
Nice to have
- Experience with machine learning and LLM fundamentals, model architecture, or inference.
- Experience with performant ML kernels or platforms such as CUDA, Triton, FlashInfer, vLLM, SGLang, or TensorRT.
Pay
165,200.00 - 223,600.00 USD annually (base salary)
Tools
Python, C++, NKI (Neuron Kernel Interface), AWS Trainium, Neuron SDK, PyTorch, CUDA, Triton
AI focus
Enables and optimizes distributed inference for LLMs and other machine-learning models on AWS Trainium, including kernel development, model tuning, and performance optimization.
Model-written summary · the original posting is authoritative