Contra Labs
Strategic Project Lead
$150K–$200K
San Francisco · Remote · Full-time · Posted Jul 1, 2026
- Experience
- 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
About the job
Leads end-to-end client projects that create and evaluate datasets, benchmarks, and research for creative AI tools.
What you'll do
- Design annotation protocols, rubrics, and evaluation studies; analyze results and present actionable findings.
- Manage delivery across clients, creative evaluators, and engineering, including scope, staffing, quality, timelines, and client relationships.
- Improve repeatable project processes and tooling, and develop case studies and benchmark publications.
What they ask for
- Based in San Francisco, California.
- 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
- Experience with data labeling, annotation, human-in-the-loop systems, or AI and ML evaluation workflows.
- Track record managing several concurrent projects, including annotation, evaluation, or QA work.
Nice to have
- Master's or PhD in HCI, cognitive science, psychology, design research, or a related field; equivalent depth from industry is also a plus.
- Statistical methods for evaluation, RLHF, LLM evaluation, benchmark design, ethnography, UX research, design-study methods, or startup experience.
Pay
$150,000 to $200,000 USD (base salary); $150K – $200K (base salary)
Tools
AI, ML, RLHF, LLM
AI focus
Supports creative AI through post-training dataset creation, benchmarking, and ongoing evaluation; develops methods for assessing creative and multimodal outputs.
Model-written summary · the original posting is authoritative