Institute of Foundation Models
Data Engineer
$150K–$450K
Sunnyvale, CA · On-site · Full-time · Posted Apr 9, 2025
- Experience
- Extensive experience required; no specific number of years stated.
- Education
- Bachelor's degree required in Computer Science, Data Science, Engineering, or a related technical field; Master’s degree or PhD degree or equivalent experience preferred.
About the job
Collects, curates, and prepares large-scale datasets for NLP research by building web crawlers, refining LLM outputs, and implementing data pipelines.
What you'll do
- Collect, curate, preprocess, and deliver datasets to researcher specifications.
- Develop web crawling solutions, APIs, automated workflows, and scalable data pipelines.
- Refine and evaluate LLM outputs and document dataset and pipeline methods.
What they ask for
- Bachelor's degree in Computer Science, Data Science, Engineering, or a related technical field.
- Extensive experience in data engineering, data processing, and automation using Python.
- Proficiency designing and deploying web crawlers, automated data extraction, and processing pipelines.
- Understanding of data structures, algorithms, databases, SQL, performance optimization, and cloud infrastructure or distributed data processing frameworks.
Nice to have
- Master’s degree or PhD degree or equivalent experience in Computer Science, Data Engineering, or related technical fields.
- Experience supporting NLP or AI research, working with LLMs, or refining AI-generated data.
Pay
$150,000 – $450,000/year USD base salary
Tools
Python, SQL, AWS, Spark, Kafka, Kubernetes
AI focus
Prepare high-quality datasets for NLP research and training or benchmarking, including refining and evaluating LLM-generated content.
Model-written summary · the original posting is authoritative