Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts
Sparse Mixture-of-Experts (MoE) models have become an important approach for scaling Large Language Models (LLMs), but their inference efficiency depends strongly on expert activation patterns. Speculative decoding (SD) accelerates autoregressive generation by verifying multiple draft tokens in parallel, yet existing draft selection strategies primarily optimize acceptance likelihood. In large-scale MoE models, however, selecting draft tokens also determines the union of experts activated during verification. We observe that confidence-driven SD can introduce \textit{expert scattering}: high-p
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 56%lightseekorg/TorchSpec →
- PossiblePossibly related (embedding) · 50%sgl-project/SpecForge →
- PossiblePossibly related (embedding) · 49%[Research] JetSpec: Speculative Decoding with Parallel Tree Drafting Enables up to 9.64x Lossless LLM Inference Speedup with more than 1000TPS →
- PossiblePossibly related (embedding) · 48%DSpark: Speculative decoding accelerates LLM inference [pdf] →
- PossiblePossibly related (embedding) · 48%thu-pacman/chitu →
- LinkedLinked via arxiv author · 85%Jincheng Xie →
“Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts”
- LinkedLinked via arxiv author · 85%Runheng Liu →
“Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts”
- LinkedLinked via arxiv author · 85%Heyan Huang →
“Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts”