Software Engineer - Inference Performance

What you'll do

• Implement inference techniques and optimize inference end to end, from kernels and memory to scheduling, routing, and caching.
• Improve GPU utilization and tokens per GPU-hour; build benchmarks and communicate latency, throughput, and cost tradeoffs.
• Bring up model architectures on new hardware and contribute to open-source inference engines.

What they ask for

• Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field.
• Experience with one or more general-purpose programming languages, such as Python or C++.
• Familiarity with LLM optimization techniques and strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM.
• Demonstrated interest and experience in LLMs; deep understanding of GPU architecture.

Nice to have

• Experience improving software performance, contributing to inference engines, or working on large-scale distributed serving.
• GPU kernel optimization or production experience with quantization or speculative decoding.

Pay

$180K – $360K USD base salary

Tools

Python, C++, PyTorch, TensorRT, TensorRT-LLM, vLLM, SGLang, CUDA

AI focus

LLM inference performance engineering, including quantization, speculative decoding, KV-cache management, model serving, and GPU optimization.