Research Scientist, Clinical AI Evaluation Lead

What you'll do

• Define launch-readiness standards and produce evidence packages for Health GenAI launch decisions.
• Build clinical benchmarks and evaluation methods, including clinician review, to assess model quality, safety, fairness, and uncertainty.
• Partner with product, engineering, clinical research, regulatory, and legal teams; guide research, publish findings, and set technical direction.

What they ask for

• 10+ years of experience in AI/ML or quantitative research, including 3+ years designing and running evaluations of LLMs or foundation models.
• 3+ years applying measurement science to generative AI, including building benchmarks, validating graders, and quantifying uncertainty.
• Experience applying ML or GenAI to healthcare with clinical experts, including a system deployed to real users via a real-world clinical confirmation study.
• Strong publication record at top-tier venues or high-impact peer-reviewed journals.

Nice to have

• Experience with clinical validation, regulated medical device software (SaMD), or FDA submissions.
• Experience evaluating production models at consumer scale, including online metrics, safety regression, and adversarial testing.

Pay

$184,000/year to $257,000/year (base salary)

Tools

LLMs, foundation models, machine learning, GenAI, electronic health records, SaMD

AI focus

Develops and evaluates health generative AI, including clinical benchmarks, safety and factuality assessments, calibration, fairness, and launch evidence.