Inworld AI
Staff / Principal Software Engineer, Inference Platform - USA
$280K–$350K
Mountain View, California, USA · Hybrid · Principal · Full-time · Posted Mar 17, 2026
- Experience
- 8+ years.
About the job
Design, build, and operate the core platform for real-time AI inference, including customer-facing services, model deployment, and GPU capacity management.
What you'll do
- Build core platform services, APIs, tenant isolation, metering, and quotas.
- Allocate and schedule GPU capacity and manage customer models from weights through production serving.
- Write production code, lead technical design, and operate services by setting SLOs and responding to incidents.
What they ask for
- 8+ years building backend or distributed systems in Go, Rust, Java or C++.
- Designed and shipped a multi-service system end to end.
- Deep understanding of distributed systems, including consistency, failure modes, queuing, load balancing, and capacity modelling.
- Experience building a platform or developer product with external customers; clear technical writing.
Nice to have
- Schedulers or resource managers for scarce hardware; LLM inference serving or ML training infrastructure.
- Multi-tenant systems with usage-based billing or contractual SLAs; Kubernetes, Terraform and GitOps (ArgoCD).
Pay
$280,000 - $350,000 USD base salary
Tools
Go, Rust, Java, C++, vLLM, SGLang, TensorRT-LLM, Kubernetes
AI focus
Build and operate real-time AI inference infrastructure, including model deployment and serving.
Model-written summary · the original posting is authoritative