Firecrawl
Research Engineer - Benchmarks
$250K–$290K
San Francisco HQ +1 · Hybrid · Full-time · Posted Sep 29, 2026
- Location
- San Francisco HQ
Toronto Hub - Experience
- 4+ years in ML, research engineering, or data engineering, with evaluation or benchmark work shipped.
- Visa sponsorship
- Candidates must already be authorized to work in the US or Canada. In the US, H-1B transfers, F-1 OPT and STEM OPT, TN (Canada and Mexico), and E-3 (Australia) are supported; new H-1B sponsorship is not available for candidates outside the US. Canadian work permit sponsorship may be considered case by case through the Toronto Hub.
About the job
Build and publish independent, weekly benchmarks of data providers, then use the results to guide Alexandria’s provider selection.
What you'll do
- Design and run provider benchmarks against verified ground truth; maintain datasets, scoring methods, and reproducible automated tests.
- Publish weekly results and public leaderboards with marketing, measuring accuracy, coverage, freshness, speed, and cost.
- Feed benchmark results into Alexandria’s provider-selection process and expand benchmark categories.
What they ask for
- Shipped evaluations or benchmarks and can explain why the results are trustworthy.
- Strong Python and API integration skills, including working with messy APIs, rate limits, and inconsistent schemas.
- Experience building test sets and scoring methods, including sampling, labeling, and inter-rater agreement.
- Uses AI heavily and communicates findings clearly to engineers, marketers, and vendors.
Nice to have
- Experience at a frontier lab, data company, third-party benchmark organization, or public benchmark project; open-source contributions welcome.
Pay
$250,000–$290,000 USD/year (SF) base salary; $210,000–$224,000 CAD/year (Toronto) base salary
Tools
Python, Apollo, FullEnrich, DataLegion, Built In, ZipRecruiter
AI focus
Use AI in work processes, including deciding when to trust an LLM judge; connect benchmark results to Alexandria’s AI-driven provider selection.
Model-written summary · the original posting is authoritative