Thinking Machines Lab
Site Reliability Engineer, Post Training
$300K–$350K
San Francisco +1 · On-site · Full-time · Posted Aug 31, 2026
- Location
- San Francisco
New York - Experience
- 4+ years; no alternative stated.
- Visa sponsorship
- Visa sponsorship is offered; success is not guaranteed for every candidate or role.
About the job
Own the reliability and performance of post-training and reinforcement learning systems, clusters, and pipelines, helping research teams run and iterate on large-scale model training.
What you'll do
- Monitor, debug, and improve training runs across infrastructure and software; drive failures to root cause.
- Build monitoring, alerting, recovery, and internal tooling to improve reliability and cluster utilization.
- Support researchers during active runs, participate in on-call, and turn recurring incidents into infrastructure fixes.
What they ask for
- 4+ years operating large-scale distributed systems in production as a production engineer, SRE, or infrastructure engineer.
- Track record debugging complex distributed-systems failures, including networking, hardware, kernel, or scheduler issues.
- Strong software engineering skills in Python and/or Go/C++; solid grounding in Linux internals and networking fundamentals.
- Comfort owning production systems and participating in on-call rotations.
Nice to have
- Experience operating GPU or TPU training clusters at scale.
- Familiarity with post-training/RL techniques and infrastructure, distributed training frameworks and schedulers, high-performance networking, or ML-training observability.
Pay
$300,000- $350,000 USD (base salary)
Tools
Python, Go, C++, Linux, PyTorch, Ray, Slurm, Kubernetes
AI focus
Reliability engineering for post-training and reinforcement learning model training systems and workloads.
Model-written summary · the original posting is authoritative