Filevine
QA Analyst - LOIS for Word
$105K–$150K
San Francisco · Remote · Full-time · Posted Jun 1, 2026
- Experience
- 2+ years of QA experience.
About the job
Own quality for LOIS for Word by developing its QA strategy and automation and evaluating AI-generated legal drafting and product behavior.
What you'll do
- Define the product’s test strategy, QA process, and playbook; partner with product and engineering early in feature development.
- Evaluate AI output quality and document-edit accuracy and formatting; build evaluation sets and automated CI/CD guardrails.
- Write reproducible bug reports and translate legal-domain customer workflows into test coverage.
What they ask for
- 2+ years of QA experience, including ownership of test strategy for a product or major feature area.
- Experience building QA process and automation from scratch in a fast-moving environment.
- Strong analytical and problem-solving skills, including assessing non-deterministic systems and AI output quality.
- Experience with bug tracking tools and test management workflows; excellent written communication.
Nice to have
- Experience testing AI or LLM-based products, including output accuracy and evaluation design.
- CI/CD experience, UX sensibility, legal technology experience or comfort reading legal contracts, or programming beyond test automation.
Pay
$105,000 - $150,000 (base salary)
Tools
LOIS for Word, CI/CD
AI focus
Evaluate legal AI output for accuracy, citation reliability, and appropriately scoped reasoning; design evaluation sets and regression checks for non-deterministic output.
Model-written summary · the original posting is authoritative