Research Engineer / Scientist - Storage for LLM

What you'll do

• Build and maintain distributed KV cache systems for transformer states across GPUs or nodes, including sharding, eviction, replication, and consistency.
• Integrate caching with inference and serving pipelines; optimize low-latency access to long-context inputs and reused embeddings.
• Measure system performance and improve caching algorithms to reduce inference response time and compute costs.

What they ask for

• PhD in Computer Science, Applied Mathematics, Electrical Engineering, or a related technical field.
• Strong understanding of transformer internals and KV caching for autoregressive decoding.
• Experience with distributed systems, memory management, and low-latency serving.
• Familiarity with high-performance compute environments and proficiency in a systems language such as C++, Rust, Go, or CUDA.

Nice to have

• Experience building LLM inference-serving systems, such as vLLM, SGLang, FasterTransformer, DeepSpeed, or Hugging Face Text Generation Inference.
• Experience with memory hierarchy and GPU communication optimization, or exposure to cache-aware scheduling, batching, and prefetching.

Tools

RPC, gRPC, CUDA, NVIDIA GPUs, TensorRT, Triton Inference Server, C++, Rust

AI focus

Develops and optimizes KV caching for transformer-based LLM inference, including reuse of attention states and prompt embeddings.