Contra Labs
Strategic Project Lead
$150K–$200K
New York City (Onsite) · On-site · Full-time · Posted Jun 8, 2026
- Experience
- 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
- Education
- A Master's or PhD in HCI, cognitive science, psychology, design research, or a related field is a plus; equivalent depth from industry is also a plus.
About the job
Owns end-to-end client projects that create and evaluate AI training data and benchmarks for creative and multimodal outputs.
What you'll do
- Design annotation protocols, rubrics, and evaluation methods; oversee project scope, staffing, timelines, client relationships, and quality.
- Analyze findings and deliver reports, recommendations, workshops, case studies, and benchmark publications.
- Improve repeatable project processes and partner with engineering on supporting tools.
What they ask for
- Based in NYC and able to work in the Williamsburg office five days a week.
- Experience with data labeling, annotation, human-in-the-loop systems, or AI/ML evaluation workflows.
- Track record managing several concurrent projects, including annotation, evaluation, or QA work.
- A point of view on creative quality and how to define it.
Nice to have
- Strong qualitative and quantitative research background; a relevant Master's or PhD, or equivalent depth from industry.
- Statistical evaluation methods, RLHF, LLM evaluation, benchmark design, ethnography, UX research, design-study methods, or startup/0-to-1 experience.
Pay
$150,000 to $200,000 USD base salary; equity (amount not stated)
Tools
AI, ML, RLHF, LLM
AI focus
Supports AI post-training dataset creation, benchmarking, and evaluation of creative and multimodal outputs.
Model-written summary · the original posting is authoritative