Causal Labs
Member of Technical Staff — Data Infrastructure
San Francisco +1 · On-site · Staff · Full-time · Posted Jul 19, 2026
- Location
- San Francisco
Singapore
About the job
Builds and operates the storage, compute, ingestion, and loading platform that makes large-scale physical-observation datasets available for AI training and research.
What you'll do
- Design and operate petabyte-scale storage and data layouts for batch and real-time queries.
- Own shared compute and orchestration, and optimize data loading into training up to the tensor boundary.
- Build data lifecycle systems for cataloging, deduplication, lineage, search, reproducibility, quality, and monitoring.
What they ask for
- Demonstrated experience building large-scale data pipelines and distributed compute systems.
- Knowledge of data ingestion, storage, and loading methods and tools, including relevant file formats and storage systems.
- Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
- Understanding of how data-loading throughput affects large-scale training, with experience optimizing it.
Tools
Spark, Ray, Beam, Parquet, Zarr, Delta Lake
AI focus
Builds data infrastructure supporting a Large Physics foundation Model intended to predict physical systems and identify actions to control them, starting with weather.
Model-written summary · the original posting is authoritative