DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding
Mixture-of-Experts (MoE) models have been widely adopted in real-time interactive applications such as coding assistants, real-time audio-video interaction systems. To meet the extremely low response latency requirements of these scenarios, practitioners commonly employ small-batch decoding, under which MoE inference becomes memory-bound and is severely bottlenecked by expert weight loading. However, this bottleneck has received limited attention, and existing solutions such as post-training weight compression or fine-grained expert design during pre-training either degrade model accuracy or i
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 54%H64LM: A 249M-parameter Mixture-of-Experts Transformer built from scratch in PyTorch [P] →
- PossiblePossibly related (embedding) · 51%Adaptive Mixture of Experts Gate (AMG) [R] →
- LinkedLinked via arxiv author · 85%Zewen Jin →
“DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding”
- LinkedLinked via arxiv author · 85%Shen Fu →
“DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding”
- LinkedLinked via arxiv author · 85%Zeping Duan →
“DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding”
- LinkedLinked via arxiv author · 85%Shannon Wang →
“DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding”
- LinkedLinked via arxiv author · 85%Weihao Wu →
“DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding”
- LinkedLinked via arxiv author · 85%Chengjie Tang →
“DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding”