Software Engineer-AI/ML, Inference Team - AWS Neuron

What you'll do

• Implement and upstream inference features such as continuous batching, paged attention, quantization, and distributed inference in frameworks including vLLM and SGLang.
• Expand out-of-the-box model support and automate model enablement, testing, benchmarking, and release workflows.
• Collaborate across engineering teams to deliver reliable, accurate, scalable, and efficient model inference.

What they ask for

• 3+ years of non-internship professional software development experience.
• 3+ years of non-internship experience designing or architecting new and existing systems, including design patterns, reliability, and scaling.
• 2+ years designing and developing large-scale software applications, tools, systems, or services using C#, C++, Java, or Perl.
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field; knowledge of Machine Learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques.

Nice to have

• Experience debugging, profiling, and applying software engineering best practices in large-scale systems.
• Experience deploying LLMs in production on AI acceleration hardware, using inference platforms such as vLLM, SGLang, or TensorRT, or developing kernels with CUDA or Triton.

Pay

143,700.00 - 194,400.00 USD annually (base salary)

Tools

AWS Neuron, vLLM, SGLang, C#, C++, Java, Perl, CUDA

AI focus

Develop and optimize LLM inference serving on AWS Neuron, including model enablement, performance optimization, and production deployment.