Amazon
Software Engineer II - AI/ML, Neuron Inference, Neuron Inference
$165K–$224K
Cupertino, CA · Full-time · Posted Oct 30, 2026
- Experience
- 3+ years of non-internship professional software development experience; preferred: 3+ years across the full software development life cycle.
- Education
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.
About the job
Develop distributed inference support and optimize large language models and ML workloads for AWS Trainium accelerators within the Neuron SDK.
What you'll do
- Build and tune distributed inference solutions, models, and high-performance kernels for AWS Trainium.
- Profile and optimize system performance, including implementing fusion, sharding, tiling, and scheduling improvements.
- Develop model onboarding infrastructure, test and deploy software, and work with customers and cross-functional teams on model enablement.
What they ask for
- 3+ years of non-internship professional software development experience
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
- Knowledge of Python and/or C++ programming
Nice to have
- Experience with Machine Learning and Large Language Model fundamentals, including model execution optimization
- Experience with CUDA or ML/low-level kernels; vLLM, SGLang, TensorRT, PyTorch, JIT compilation, or AOT tracing
Pay
USD 165,200.00 - 223,600.00 annually (base salary)
Tools
Python, C++, AWS Trainium, Neuron SDK, PyTorch, CUDA, vLLM, SGLang
AI focus
Develop and optimize distributed inference for LLMs and other ML models on AWS Trainium, including model enablement, kernel development, and performance tuning.
Model-written summary · the original posting is authoritative