Hims & Hers
Sr. Staff Machine Learning Systems Engineer
$240K–$265K
US - Remote · Remote · Senior Staff · Full-time · Posted Aug 18, 2026
- Experience
- 10+ years in ML infrastructure, data engineering, or evaluation/testing systems.
About the job
Owns the data pipelines and evaluation systems that determine whether AI models are effective and safe to deploy in regulated healthcare.
What you'll do
- Set technical direction for evaluation methodology, judges and scorers, statistical regression gates, and failure tracking.
- Design and scale data pipelines for ingestion, transformation, dataset versioning, labeling, and calibration.
- Lead cross-team AI evaluation and red-team initiatives, deliver platform improvements, and mentor engineers.
What they ask for
- 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems.
- Hands-on experience designing and calibrating LLM judges/scorers, statistical regression testing, human-label agreement measurement, and adversarial/red-team evaluations.
- Hands-on experience with dataset versioning, feature and benchmark pipelines, labeling and calibration workflows, and high-throughput data systems.
- Strong Python and statistical fluency; experience leading multi-team projects and mentoring engineers.
Nice to have
- Experience with Databricks, MLflow, Unity Catalog, or similar platforms.
- Experience building reporting tools for non-engineering users or working in a regulated industry.
Pay
$240K – $265K USD base salary
Tools
Python, Databricks, MLflow, Unity Catalog, Slack
AI focus
Build and operate AI evaluation infrastructure, including LLM judges and scorers, statistically sound regression gates, and adversarial/red-team testing to assess model effectiveness and safety before deployment to patients.
Model-written summary · the original posting is authoritative