ByteDance
Tech Lead Software Engineer - AI Compute Infrastructure
San Jose, California · Full-time · Posted Oct 28, 2026
- Experience
- 5 years of relevant experience; Ph.D. with strong systems/ML publications also considered.
- Education
- B.S./M.S. in Computer Science, Computer Engineering, or related fields; Ph.D. with strong systems/ML publications also considered.
About the job
Build and operate cloud-native, GPU-optimized infrastructure and orchestration systems for large-scale LLM inference.
What you'll do
- Design and build high-performance, scalable container-based cluster management and orchestration systems.
- Architect GPU and AI accelerator infrastructure for cost-efficient, secure ML platforms.
- Collaborate on LLM inference solutions and write production-ready code.
What they ask for
- B.S./M.S. in Computer Science, Computer Engineering, or a related field with 5 years of relevant experience; Ph.D. with strong systems/ML publications also considered.
- Strong understanding of large model inference, distributed and parallel systems, and/or high-performance networking systems.
- Hands-on experience building cloud or ML infrastructure, such as resource management, scheduling, request routing, monitoring, or orchestration.
- Knowledge of Docker and Kubernetes, and proficiency in Go, Rust, Python, or C++.
Nice to have
- Experience with large-scale cluster management, workload scheduling, GPU orchestration, scaling, or isolation in production.
- Experience with CUDA or inference engines such as vLLM, SGLang, or TensorRT-LLM.
Tools
Kubernetes, Ray, Docker, Go, Rust, Python, C++, vLLM
AI focus
Develop infrastructure for LLM inference and AI workloads, including GPU and AI accelerator orchestration and integration with inference engines.
Model-written summary · the original posting is authoritative