Kvcache

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

[Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

vMLX - JANGTQ Uber Compressed MLX Models - L2 Disk Cache (survives restart) + L1 Paged (super fast ttft) + Hybrid SSM Scheduler + Cont Batching + etc!

Persist and reuse KV Cache to speedup your LLM.