Inferact
Member of Technical Staff, Inference
Remote · Remote · Staff · Full-time · Posted Sep 4, 2026
- Education
- Bachelor's degree or equivalent experience in computer science, engineering, or similar.
- Visa sponsorship
- Visa sponsorship is offered on a case-by-case basis.
About the job
Optimize vLLM and other inference runtimes to serve LLMs and diffusion models efficiently across diverse hardware and architectures.
What you'll do
- Optimize model execution and inference performance across hardware and architectures.
- Implement inference techniques and model architectures based on research papers.
- Contribute performant, maintainable code and debug complex ML codebases.
What they ask for
- Bachelor's degree or equivalent experience in computer science, engineering, or similar.
- Deep understanding of transformer architectures and their variants.
- Strong Python programming skills and experience with PyTorch internals.
- Experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, or TGI.
Nice to have
- Understanding of KV-cache memory management, prefix caching, and hybrid model serving; familiarity with RL frameworks and algorithms for LLMs.
- Experience with multimodal inference or contributions to open-source ML or systems infrastructure projects.
Tools
vLLM, Python, PyTorch, TensorRT-LLM, SGLang, TGI, verl, OpenRLHF
AI focus
Develop and optimize inference serving for LLMs and diffusion models, including evolving architectures such as mixture-of-experts, multimodal, and agentic models.
Model-written summary · the original posting is authoritative