Applied AI Engineer Intern - Advancing Agent Quality

What you'll do

• Design realistic agent environments and task suites; develop LLM judges and benchmark agents against frontier models.
• Analyze agent trajectories and failure modes, build diagnostic tools, and feed improvements into prompts, harnesses, and architecture.
• Create datasets and reward signals from interactions, support model fine-tuning and post-training, and contribute research insights to the Think Tank.

What they ask for

• Currently enrolled in a master’s or PhD program, or a postdoctoral or early-career academic researcher, in a related field.
• 1 year of research, project, or professional experience with LLMs or agents; 2 years of experience with machine learning and deep learning.
• Experience in software engineering and cloud-based development.
• Experience with agent builders; ability to explain complex technical concepts to non-technical executive audiences.

Nice to have

• Hands-on experience with LLM or agent evaluation systems, such as autoraters, human-evaluation pipelines, benchmark design, or evaluation infrastructure.
• Experience with multi-agent orchestration, retrieval-augmented generation, tool-use reliability, or relevant active research or publications.

Tools

LLMs, Agents, OpenSDK, OpenAI Agent Builder

AI focus

Agent evaluation and quality; automated judging, benchmarking, failure analysis, verifier generation, and data and reward signals for fine-tuning and post-training; AI research and Think Tank analysis.