Amazon
Senior ML Software Engineer, Data Plane
$193K–$262K
Cupertino, CA · Senior · Full-time · Posted Aug 4, 2026
- Experience
- 7+ years of full software development life cycle experience.
- Education
- Bachelor's degree in computer science or equivalent.
About the job
Designs and delivers the inference data plane for large language models on custom ML accelerator hardware, optimizing performance across kernels, model execution, serving, and distributed systems.
What you'll do
- Develop and optimize compute kernels and validate LLM architectures end-to-end on custom hardware.
- Integrate accelerator backends with ML serving frameworks and build model correctness, testing, profiling, and CI/CD infrastructure.
- Own software features through design, implementation, testing, and integration; mentor engineers and lead design reviews.
What they ask for
- Bachelor's degree in computer science or equivalent.
- 7+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
- Strong proficiency in C/C++ and strong Linux systems knowledge.
- Experience developing compute kernels for GPUs, DSPs, or custom accelerators, and a track record of delivering complex software features end-to-end.
Nice to have
- Knowledge of ML and LLM fundamentals and frameworks including JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, and TensorRT.
- Experience with distributed systems, hardware simulation, LLM serving optimizations, or AI-assisted development tools.
Pay
193,300.00 - 261,500.00 USD annually (base salary)
Tools
C/C++, Linux, PyTorch, vLLM, JAX, SGLang, Dynamo, TensorRT
AI focus
Develops and optimizes LLM inference on custom ML accelerators, including kernels, model execution, serving integration, and performance validation. Preferred qualifications mention daily use of LLMs or code-generation agents for AI-assisted development.
Model-written summary · the original posting is authoritative