LM Studio
Software Engineer, Inference Runtime
$150K–$350K
New York City · Hybrid · Full-time · Posted Aug 7, 2026
- Experience
- Significant experience required; no number of years stated.
About the job
Build and optimize LM Studio’s inference runtime across on-device and cloud environments, supporting new models and improving execution on diverse hardware.
What you'll do
- Maintain and advance the inference stack; bring up new model architectures and multimodal models.
- Improve latency, throughput, memory use, and reliability across CPU, CUDA, Metal, Vulkan, and ROCm runtimes; develop runtime capabilities for loading, batching, scheduling, caching, and distributed execution.
- Benchmark and diagnose correctness and performance issues, and contribute improvements upstream to open-source projects.
What they ask for
- Significant experience building production ML systems, inference runtimes, or performance-sensitive infrastructure.
- Strong programming ability in Python and C++; deep understanding of transformer architectures and model inference.
- Experience profiling CPU or GPU workloads and reasoning about compute, memory, synchronization, and data movement.
- Experience with PyTorch and inference systems such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM.
Nice to have
- Past contributions to open-source inference runtime projects such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM.
Pay
$150K – $350K USD base salary
Tools
Python, C++, PyTorch, llama.cpp, MLX, ExecuTorch, vLLM, TensorRT-LLM
AI focus
Develop and optimize AI model inference runtimes, including support for open-weight and multimodal models across on-device and cloud environments.
Model-written summary · the original posting is authoritative