Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs
Serverless multi-model LLM systems multiplex popularity-skewed model catalogs over shared GPU pools, yet typically schedule each request independently. Tool-using agents break this abstraction: a session repeatedly calls an LLM across short tool gaps, carries a long reusable KV prefix, and is judged by session completion time (SCT). Load-only routing can separate a continuation from both its model and KV state, while round-based model multiplexing can delay even a correctly placed continuation until the target model's next slot. Both failures are especially costly for hundred-billion-parameter
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 84%tensorflow/serving →
“Fuzzy title match (0.92): “Talaria: Session-Aware Serverless Serving of Hundred-Billion” ≈ “tensorflow/serving””
- LinkedLinked via arxiv author · 85%Utopia Meng →
“Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs”
- LinkedLinked via arxiv author · 85%Unicornt Zhao →
“Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs”
- LinkedLinked via arxiv author · 85%Derek Li →
“Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs”
- LinkedLinked via arxiv author · 85%Goalen Gao →
“Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs”
- LinkedLinked via arxiv author · 85%Frank Du →
“Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs”