Speed

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

TokenSpeed is a speed-of-light LLM inference engine.

A PyTorch native library for training speculative decoding models