Exa
Software Engineer, Knowledge Engine – Data Infrastructure
$180K–$350K
San Francisco, California · On-site · Full-time · Posted Jun 14, 2026
About the job
Build the data pipelines that turn web crawls and other sources into a reliable, continuously updated knowledge graph.
What you'll do
- Ingest and transform web, vendor, API, and other data into claims with provenance, timestamps, and confidence.
- Build versioned, replayable ingestion and incremental processing systems that handle corrections and keep graph data consistent.
- Develop entity resolution, truth discovery, and evaluation measures for precision, completeness, and data quality.
What they ask for
- Production experience building and operating large-scale distributed data pipelines over petabytes of structured and unstructured data, using Spark with a lakehouse format.
- Deep understanding of incremental processing, including change data capture, idempotent writes, backfills, and reprocessing.
- Strong Python skills and proficiency in at least one systems language.
- Ability to read and tune Spark query plans.
Nice to have
- Experience with entity resolution, record linkage, knowledge graphs, or ontologies.
- Experience with information extraction or truth discovery at scale; Lance or other columnar/vector-native formats.
Pay
$180K – $350K USD base salary
Tools
Spark, Delta Lake, Iceberg, Hudi, Python, Lance
Model-written summary · the original posting is authoritative