INFERENCE ENGINEER

What you'll do

• Build, harden, and operate high-throughput production inference systems; own throughput, latency, cost-per-token, and reliability.
• Profile workloads and integrate inference optimizations, including quantization, custom kernels, scheduling, and memory management.
• Design observability, plan capacity and autoscaling, load-test systems, and resolve incidents with systemic follow-up fixes.

What they ask for

• 3+ years building production-grade, large-scale serving infrastructure.
• Strong distributed systems experience, including on-call responsibility.
• Experience with GPU-accelerated inference at scale across multi-GPU, multi-node, batched, and streaming workloads.
• Fluent Python and ability to read and write systems-level code in at least one of C++, CUDA, ROCm, or Triton.

Nice to have

• Open-source contributions to inference or serving frameworks; experience with mixed cloud and on-premises deployments.
• Familiarity with hardware-aware optimization such as memory hierarchy, NCCL/RDMA, or NUMA; background in compilers, runtimes, or accelerator software stacks.

Tools

Python, C++, CUDA, ROCm, Triton, NCCL/RDMA, NUMA

AI focus

Serves large models in production for a company building autonomous research agents that propose, run, and analyze machine learning experiments.