Senior Research Engineer / Scientist - Storage for LLM

What you'll do

• Build and maintain distributed caches for transformer attention states and reused embeddings across GPUs or nodes.
• Optimize cache access, eviction, consistency, sharding, and replication for multi-tenant inference workloads.
• Integrate the cache with inference pipelines and evaluate performance to reduce response time and compute costs.

What they ask for

• PhD in Computer Science, Applied Mathematics, Electrical Engineering, or a related technical field.
• Strong understanding of transformer internals and KV caching in autoregressive decoding.
• Experience with distributed systems, memory management, and low-latency serving.
• Familiarity with high-performance compute environments and proficiency in C++, Rust, Go, or CUDA.

Nice to have

• Experience building LLM inference-serving systems such as vLLM or SGLang.
• Experience with memory hierarchy optimization, GPU communication, or cache-aware serving strategies.

Tools

RPC, gRPC, CUDA, NVIDIA GPUs, TensorRT, Triton Inference Server, C++, Rust

AI focus

Develop GPU-aware distributed KV caching for transformer-based LLM inference, including reuse of attention states and prompt embeddings.