Cerebras
Senior Software Engineer, ML API – AI Inference
Toronto, CAN +1 · Hybrid · Senior · Full-time · Posted Oct 27, 2026
- Location
- Toronto, CAN
Sunnyvale, CA - Experience
- 5+ years of software engineering experience.
- Education
- Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.
About the job
Build and operate the ML API, model integrations, and request-routing services for Cerebras’s heterogeneous AI inference platform.
What you'll do
- Design, implement, test, and operate inference API features and model integrations across GPU prefill and Cerebras decode.
- Lead multi-component delivery, including design documentation, implementation coordination, validation, and rollout.
- Improve inference performance, correctness, and production reliability through testing, observability, and incident resolution.
What they ask for
- 5+ years of software engineering experience, including independent ownership of production services, distributed systems, or substantial software components.
- Strong Python programming ability and proficiency in Go, C++, Rust, or another language used to build production services.
- Experience building production APIs and integrating software across service, framework, runtime, or infrastructure boundaries.
- Experience operating Linux services using containers, Kubernetes or comparable orchestration systems, and CI/CD.
Nice to have
- Experience building or operating production ML or LLM inference APIs, including OpenAI-compatible interfaces.
- Experience with ML systems projects such as vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, or TensorRT-LLM.
Tools
Python, Go, C++, Rust, vLLM, PyTorch, Hugging Face, Kubernetes
AI focus
Develop inference APIs and serving systems for foundation models, including model integration, tool use, structured and multimodal inputs, and disaggregated GPU prefill with Cerebras decode.
Model-written summary · the original posting is authoritative