Software Engineer - AI Compute Infrastructure

What you'll do

• Design and build scalable, resilient container-based cluster management and orchestration systems.
• Architect cost-efficient, secure GPU and AI accelerator infrastructure for machine learning platforms.
• Develop inference solutions with LLM engines and integrate open-source and systems research into production.

What they ask for

• B.S./M.S. in Computer Science, Computer Engineering, or a related field with 2+ years of relevant experience; Ph.D. with strong systems/ML publications also considered.
• Strong understanding of large model inference, distributed and parallel systems, and/or high-performance networking systems.
• Hands-on experience building cloud or ML infrastructure, such as resource management, scheduling, request routing, monitoring, or orchestration.
• Knowledge of Docker and Kubernetes, and proficiency in at least one of Go, Rust, Python, or C++.

Nice to have

• Experience with large-scale cluster management, workload scheduling, GPU orchestration, scaling, or isolation in production.
• Hands-on GPU programming or inference engine experience; familiarity with public cloud ML platforms and distributed ML systems.

Tools

AIBrix, Kubernetes, Ray, Docker, Go, Rust, Python, C++

AI focus

Build GPU-optimized infrastructure for LLM inference and machine learning workloads; deliver inference solutions using vLLM, SGLang, TensorRT-LLM, and other LLM engines.