Senior Staff LLM Inference Engineer

What you'll do

• Prototype LLM inference use cases and proof-of-concept systems for heterogeneous deployments.
• Develop and optimize kernels, operators, quantization, sparsity, batching, runtimes, and serving frameworks.
• Contribute to distributed inference and collaborate with hardware, firmware, compiler, product, and business development teams.

What they ask for

• Bachelor’s degree in Computer Science, Electrical Engineering, or a related field, and 10+ years of relevant engineering experience or equivalent demonstrated experience.
• Strong proficiency in Python and C/C++.
• Hands-on LLM inference optimization experience, including attention kernels, KV cache, batching, and quantization (INT8/FP8/INT4).
• Contributor-level experience with a major inference framework and familiarity with GPU kernel programming (CUDA/Triton) and performance profiling tools.

Nice to have

• Master’s or PhD in a related field, with 6+ years of relevant industry experience.
• Experience with heterogeneous compute, custom silicon, distributed or production inference, open-source ML systems, or advanced serving techniques.

Pay

$195K – $285K USD base salary

Tools

Python, C/C++, vLLM, SGLang, TensorRT-LLM, ONNX Runtime, CUDA, Triton

AI focus

LLM inference engineering and optimization across heterogeneous hardware, including kernels, quantization, distributed serving, and deployment prototypes.