Backend Inference Framework Engineer Graduate (AML Inference) - 2027 Start

What you'll do

• Design and implement model inference service architecture for availability, scalability, and performance.
• Develop and optimize inference framework modules, including scheduling, monitoring, alerting, and canary release.
• Evaluate emerging inference technologies and improve distributed service architecture and technical standards.

What they ask for

• Completing or recently completed a Bachelor's or Master's degree in Software Development, Computer Science, Computer Engineering, or a related discipline.
• Solid C/C++ skills, knowledge of data structures and algorithms, and familiarity with basic Linux commands.
• Understand multithreaded concurrency, including threads, synchronization locks, and thread pools; identify common concurrency issues and perform basic performance tuning.
• Experience with high-concurrency distributed service R&D and familiarity with service latency and resource optimization.

Nice to have

• Project experience and source-code understanding in Redis, RocksDB, BRPC, or GRPC.
• Understanding of GPU operation and relevant experience optimizing GPU service resource management and control.

Tools

Linux, C/C++, Redis, RocksDB, BRPC, GRPC, GPUs

AI focus

Builds and optimizes inference infrastructure for large, complex AI models.