OpenAI
Data Engineer
$235K–$385K
San Francisco +1 · Hybrid · Full-time · Posted Sep 26, 2023
- Location
- San Francisco
Mountain View - Experience
- 3+ years as a data engineer and 8+ years of software engineering experience, including data engineering.
About the job
Build and manage data pipelines and canonical datasets that support product analytics, safety systems, business decisions, and model training.
What you'll do
- Design and manage pipelines integrating user event data into the data warehouse, and develop canonical datasets for product metrics.
- Collaborate with teams across the company to understand data needs and provide solutions; contribute to data architecture and engineering decisions.
- Build fault-tolerant data ingestion and processing systems, and ensure data security, integrity, and compliance.
What they ask for
- Proficiency in at least one of Python, Scala, or Java.
- Experience with distributed processing technologies and frameworks, such as Hadoop or Flink, and distributed storage systems such as HDFS or S3.
- Expertise with an ETL scheduler such as Airflow, Dagster, Prefect, or similar.
- Solid understanding of Spark and ability to write, debug, and optimize Spark code.
Pay
$235K – $385K (base salary)
Tools
Python, Scala, Java, Hadoop, Flink, HDFS, S3, Airflow
AI focus
Supports safety systems and provides an opportunity to collaborate with researchers on training new models.
Model-written summary · the original posting is authoritative