Thinking Machines Lab
Software Engineer, Evaluation Platform / Infra
$300K–$350K
San Francisco +1 · On-site · Full-time · Posted Aug 31, 2026
- Location
- San Francisco
New York - Experience
- Two years of post-grad work experience as a software engineer or ML engineer, exclusive of internships.
- Education
- Bachelor’s degree, or equivalent practical experience, in computer science, engineering, machine learning, or a related field.
- Visa sponsorship
- Visa sponsorship is available; success is not guaranteed for every candidate or role.
About the job
Designs and builds an end-to-end platform that helps researchers author, run, reproduce, and analyze model evaluations to guide AI research and model releases.
What you'll do
- Build and maintain evaluation tooling across libraries, distributed systems, data pipelines, APIs, and user-facing applications.
- Create flexible evaluation abstractions and ensure results are reproducible and trustworthy through versioning, observability, and quality controls.
- Partner with researchers to turn bespoke evaluation workflows into self-serve systems used across teams.
What they ask for
- Bachelor’s degree or equivalent practical experience in computer science, engineering, machine learning, or a related field.
- Two years of post-grad software or ML engineering experience, excluding internships.
- Hands-on experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
- Strong software engineering fundamentals; proficiency in at least one backend programming language; and experience with data-intensive infrastructure.
Nice to have
- Experience building frameworks, SDKs, or developer tools with thoughtful abstractions and strong user experience.
- Experience with distributed job execution, workflow orchestration, sandboxed environments, or large-scale data processing.
Pay
$300,000 - $350,000 USD (base salary)
Tools
Python, Rust, React, Typescript
AI focus
Build evaluation infrastructure for large language and multimodal models, enabling researchers to assess capabilities and behaviors and use results to guide research and model releases.
Model-written summary · the original posting is authoritative