Rhoda AI
Software Engineer, Inference Infrastructure
$175K–$280K
Mountain View · Full-time · Posted May 12, 2026
- Experience
- 3+ years in ML infrastructure, MLOps, or distributed systems.
About the job
Build and operate scalable infrastructure for deploying and serving foundation-model workloads across cloud and on-prem environments.
What you'll do
- Design and operate infrastructure for model workloads across cloud and on-prem environments.
- Build Kubernetes deployment pipelines and manage GPU resource scheduling and orchestration.
- Integrate model-serving systems and develop tooling for deployment, versioning, and observability.
What they ask for
- Strong proficiency with Kubernetes and containerized deployment pipelines.
- Experience with GPU orchestration and resource scheduling across large distributed jobs.
- Experience with cloud providers and hybrid cloud/on-prem infrastructure.
- Familiarity with ML frameworks and model-serving tools; strong debugging and ownership skills.
Nice to have
- Experience with streaming systems or high-throughput data transport.
- Background in networking, low-latency systems, edge/cloud deployment, embedded inference, or cluster scheduling systems.
Pay
$175K – $280K (base salary)
Tools
Kubernetes, AWS, GCP, PyTorch, JAX, Triton, Ray Serve, TorchServe
AI focus
Infrastructure for deploying and serving large foundation models, including distributed ML workloads and inference.
Model-written summary · the original posting is authoritative