Software Engineer, Inference

What you'll do

• Operate and scale production inference systems, including Tinker's multi-tenant serving platform.
• Manage safe rollouts of models and inference optimizations; improve observability, alerting, and capacity planning.
• Lead incident response and design resilient serving systems, balancing infrastructure capacity and cost.

What they ask for

• Experience operating large-scale, latency-sensitive production systems.
• Proficiency in Python and Go or another systems language.
• Experience with observability, monitoring, and incident response for production services.
• Strong understanding of distributed systems and how they fail at scale.

Nice to have

• Experience running production inference for large language models or other large-scale ML systems.
• Experience with deployment and rollout systems, capacity planning, cost optimization, or inference-specific techniques.

Pay

$300,000 - $400,000 USD (base salary)

Tools

Python, Go, GPU, TPU

AI focus

Productionize and serve models, including large language models, using inference techniques and optimizations.