Ollama
Software Engineer, Runtime
Palo Alto · On-site · Full-time · Posted Jul 9, 2026
About the job
Build and optimize Ollama’s local runtime for loading and running open models across operating systems and hardware.
What you'll do
- Develop runtime components for model loading, memory management, quantization, scheduling, and GPU backends.
- Integrate model architectures and quantization formats, and improve startup time, time-to-first-token, and throughput.
- Ship open-source runtime features and collaborate with model labs, hardware vendors, and the community.
What they ask for
- Strong systems fundamentals and comfort with Go, C, or C++.
- Experience working close to the metal in areas such as GPU compute, inference, game engines, databases, operating systems, or networking.
- Experience profiling and optimizing real workloads.
- Comfort shipping to millions of users across varied hardware and operating systems.
Nice to have
- Experience with model quantization or GPU programming (CUDA/Metal/SYCL).
- Apple MLX experience.
Tools
Go, C, C++, CUDA, Metal, SYCL, Apple MLX
AI focus
Enable local inference of open models by supporting model architectures, quantization formats, inference performance, and hardware acceleration.
Model-written summary · the original posting is authoritative