Large Language Model Inference System Engineer Graduate (Applied Machine Learning) - 2027 Start

What you'll do

• Contribute to engineering development of the Volcano Ark MaaS inference system.
• Optimize inference performance, cost, and stability in ultra-large-scale heterogeneous inference clusters.
• Reduce inference costs using approaches including disaggregated multi-role inference, distributed KV Cache systems, heterogeneous inference, elastic computing, and multi-tenant co-located inference.

What they ask for

• Completing or recently completed a Bachelor's or Master's degree in Computer Science or a related discipline.
• Strong command of algorithms, design patterns, and data structures, with solid operating systems and computer architecture knowledge.
• Proficient in one or more programming languages, such as C++ or Python, with good coding style.
• Understand GPU architecture; be familiar with high-performance computing software stacks such as CUDA and have GPU performance analysis experience.

Nice to have

• Experience optimizing large model inference systems, including practical understanding of PD disaggregation, KV Cache systems, and multi-node inference.
• Experience in distributed network communication optimization and familiarity with Kubernetes or Ray.

Tools

C++, Python, CUDA, Kubernetes, Ray, RDMA

AI focus

Engineering and optimizing large language model inference systems, including distributed and heterogeneous inference and KV Cache systems.