PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements of the model weights and the growing KV cache. We propose PagedWeight, a novel management method for MoE LLM serving that dynamically quantizes MoE model's weights at runtime and balances expert-weight precision with the KV cache sizes. PagedWeight exposes and effectively navigates the complex tradeoff between the model's task accuracy, memory consumption, and throug
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 26%LMCache/LMCache →
“Possibly related via embedding similarity 0.56 (not asserted). Timestamp check: artifact slightly before paper (-15d).”
- PossiblePossibly related (embedding) · 57%I mapped which local LLMs actually fit each RAM tier, 8 to 128GB (open dataset) →
- FuzzySimilar title/name (fuzzy) · 84%tensorflow/serving →
“Fuzzy title match (0.92): “PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-” ≈ “tensorflow/serving””
- LinkedLinked via arxiv author · 85%Yuchen Yang →
“PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization”
- LinkedLinked via arxiv author · 85%Yifan Zhao →
“PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization”
- LinkedLinked via arxiv author · 85%Anisha Dasgupta →
“PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization”
- LinkedLinked via arxiv author · 85%Sasa Misailovic →
“PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization”
- PossiblePossibly related (embedding) · 46%SkewAdam: A tiered optimizer that cuts MoE state memory by 97% (fits a 6.7B MoE on a 40GB GPU) [R] →