Software Engineer, GPU Inference

What you'll do

• Build and maintain the GPU prefill path across APIs, serving workers, frameworks, ROCm, GPU nodes, and infrastructure.
• Establish operational practices and reliability measures for the AMD GPU fleet, including deployment, recovery, monitoring, and incident response.
• Profile and optimize inference performance, serving behavior, numerical correctness, and performance-validation tooling.

What they ask for

• 5+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong C++ and Python programming ability; hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or equivalent.
• Experience with GPU performance, distributed-system debugging, and production operations using Linux, containers, Kubernetes or comparable orchestration, observability, and CI/CD.

Nice to have

• Experience with AMD Instinct accelerators and the ROCm ecosystem.
• Experience with CUDA or contributions to open-source ML systems projects.

Tools

C++, Python, vLLM, PyTorch, ROCm, HIP, Linux, Kubernetes

AI focus

Develop and optimize GPU-accelerated prefill for AI inference, including serving large language and multimodal models; improve inference performance, model-serving behavior, and numerical correctness.