Staff Software Engineer, GPU Inference

What you'll do

• Build, deploy, and maintain the GPU prefill serving path across APIs, model-serving workers, software runtimes, GPU nodes, networking, and infrastructure.
• Establish operational practices and reliability systems for the AMD GPU fleet, including deployment, recovery, monitoring, and incident response.
• Profile and optimize inference performance, serving behavior, correctness, and regression-testing infrastructure.

What they ask for

• 8+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong C++ and Python programming skills, plus hands-on experience with a high-performance model-serving framework such as vLLM or equivalent.
• Experience with GPU performance, distributed-system debugging, and production operations involving Linux, containers, Kubernetes or comparable orchestration, observability, and CI/CD.

Nice to have

• Experience with AMD Instinct accelerators and the ROCm ecosystem, or deep CUDA experience.
• Experience optimizing disaggregated inference, multi-GPU or multi-node serving, quantization, or contributing to ML systems projects.

Tools

C++, Python, vLLM, PyTorch, ROCm, HIP, Linux, Kubernetes

AI focus

GPU-accelerated LLM and multimodal inference, including prefill serving, performance optimization, and numerical validation.