Backend Inference Runtime Engineer Graduate (AML Inference) - 2027 Start

What you'll do

• Optimize inference-engine architecture, GPU memory access, computation, and scheduling to improve throughput and reduce latency.
• Adapt inference systems to GPU/NPU architectures and develop distributed parallel strategies for large models.
• Benchmark and implement inference technologies, including compilation, parallelism, and cache optimization.

What they ask for

• Completing or recently completed a Bachelor's/Master's degree in Software Development, Computer Science, Computer Engineering, or a related technical discipline.
• Proficient in C/C++ and Python; skilled in CUDA and knowledgeable about GPU architecture, memory models, scheduling, and communication.
• Able to implement and optimize deep-learning operators and inference compilation workflows.
• Proficient with GPU performance tools such as Nsight and Profiler; able to identify and resolve inference bottlenecks.

Nice to have

• Experience implementing distributed inference solutions, including tensor, pipeline, or sequence parallelism.
• Experience with secondary development or performance optimization of vLLM, SGLang, or TensorRT-LLM.

Tools

C/C++, Python, CUDA, Nsight, Profiler, vLLM, SGLang, TensorRT-LLM

AI focus

Build and optimize large-model inference systems, including GPU execution, inference compilation, and distributed model parallelism.