alphaXiv
LLM Inference Engineer
San Francisco Bay Area · Full-time · Posted Oct 30, 2026
About the job
Deploys and optimizes large language models by building scalable inference pipelines and bringing new research techniques into production.
What you'll do
- Build and optimize LLM inference pipelines, including batching, caching, and request routing.
- Manage GPU infrastructure and integrate inference frameworks to improve cost and performance.
- Build production ML monitoring tools and collaborate with researchers to deploy new models.
What they ask for
- Experience with LLM inference frameworks such as vLLM, TGI, or TensorRT-LLM.
- Understanding of GPU architecture and optimization techniques such as CUDA and quantization.
- Proficiency in Python and comfort with systems programming.
- Experience with containerization, orchestration, and cloud infrastructure.
Tools
vLLM, TGI, TensorRT-LLM, CUDA, Python, Docker, Kubernetes
AI focus
Deploys and serves large language models, implements LLM architectures, and optimizes inference performance.
Model-written summary · the original posting is authoritative