Sr. Software Development Engineer, Inference Team - AWS Neuron

What you'll do

• Customize and optimize vLLM and SGLang, including scheduling, model execution, kernels, distributed KV cache, and speculative decoding.
• Evaluate inference research and translate it into production-ready features while influencing the technical roadmap.
• Collaborate across engineering teams and provide design leadership and mentorship.

What they ask for

• 5+ years of non-internship professional software development experience, programming experience, system design or architecture experience, and full software development life cycle experience.
• Experience as a mentor, tech lead or leading an engineering team.
• Fundamentals of machine learning models, architecture, training and inference lifecycles, and work experience optimizing model performance.

Nice to have

• Bachelor's degree in computer science or equivalent.
• Hands-on experience with PyTorch or JAX, inference frameworks such as vLLM or SGLang, or kernel development (e.g., CUDA, Triton); production LLM experience on GPUs or AI accelerator hardware is preferred.

Pay

168,100.00 - 227,400.00 USD annually (base salary)

Tools

AWS Neuron, Inferentia, Trainium, vLLM, SGLang, PyTorch, JAX, CUDA

AI focus

Optimizes large-scale model inference and LLM serving on AWS Neuron, including kernels, parallel computation, distributed KV cache, and speculative decoding.