Research Engineer

What you'll do

• Build benchmarks, labeled datasets, and evaluation tools from production workspace data, including consent and redaction processes.
• Run experiments from offline data through production deployment, owning both evaluation and shipped features.
• Assess external vendors and research partners, and publish findings when meaningful progress is made.

What they ask for

• Applied research experience in model evaluation, benchmarking, and/or failure analysis, with evaluations trusted for decision-making.
• Work samples or code demonstrating relevant work, such as evaluation frameworks, benchmark suites, failure-analysis reports, or labeling infrastructure.
• Strong technical communication with researchers and product teams.
• Comfort working with incomplete, ambiguous, and edge-case-heavy data.

Nice to have

• Familiarity with simulation, human-in-the-loop evaluation, context or memory-compression work, or memory for long-running multi-party settings.
• Familiarity with agent observability standards, such as Langsmith or similar.

Tools

Langsmith

AI focus

Research and evaluate agent proactivity, workspace memory, retrieval, context, reinforcement learning, and continual learning; build benchmarks and take validated improvements into production.