PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
Large language models (LLMs) provide strong reasoning capabilities but are expensive to serve at scale, whereas small language models (SLMs) are cheaper but less reliable on difficult problems. We introduce PyroDash, a cost-aware framework for token-level SLM-LLM collaborative inference. During generation, the SLM decides whether to request assistance by emitting a control token. A Collaborate Engine then sends the query and partial reasoning trace to a frozen LLM for completion through a single handoff. The policy is internalized in the SLM, requiring neither a separate router, LLM retraining
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 84%xorbitsai/inference →
“Fuzzy title match (0.92): “PyroDash: Cost-Efficient Token-Level Small-Large Language Mo” ≈ “xorbitsai/inference””
- LinkedLinked via arxiv author · 85%Niqi Lyu →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”
- LinkedLinked via arxiv author · 85%Pengtao Shi →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”
- LinkedLinked via arxiv author · 85%Wei Qiu →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”
- LinkedLinked via arxiv author · 85%Jianlin Zhong →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”
- LinkedLinked via arxiv author · 85%Sicong Xia →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”
- LinkedLinked via arxiv author · 85%Jianyao Ma →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”
- LinkedLinked via arxiv author · 85%Yicheng Ding →
“PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference”