Hippocratic AI
Staff Site Reliability Engineer
Menlo Park, CA · On-site · Staff · Full-time · Posted Sep 17, 2026
- Experience
- 10+ years of professional experience across site reliability/DevOps engineering and software engineering.
- Education
- Computer Science degree from a top CS program required.
About the job
Build and operate a GPU management and scheduling platform that routes inference requests and scales model replicas across a fleet of models and heterogeneous hardware.
What you'll do
- Build metrics pipelines, admission control, and autoscaling systems for GPU-backed model workloads.
- Develop cloud orchestration, infrastructure automation, and deployment pipelines; operate secure, fault-tolerant production systems.
- Maintain monitoring, logging, and alerting; resolve operational issues and mentor engineers.
What they ask for
- 10+ years of professional experience across site reliability/DevOps engineering and software engineering.
- Computer Science degree from a top CS program.
- Build orchestration and scheduling systems in Python and/or Go, and have experience using operational metrics to drive control loops.
- Experience with infrastructure automation and CI/CD, a major cloud platform, Docker and Kubernetes, and monitoring/logging stacks.
Nice to have
- Experience managing GPU fleets or heterogeneous accelerator workloads; familiarity with ML inference serving and model deployment.
- Experience with Kubernetes autoscaling internals, HIPAA and SOC 2 compliance, or HPC environments.
Tools
Python, Go, Terraform, GitLab CI/CD, AWS, GCP, Azure, Kubernetes
AI focus
Manage GPU-backed models and schedule, route, and scale inference workloads across the model fleet.
Model-written summary · the original posting is authoritative