SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
LLM scheduling is critical to serving, yet it remains unclear how well existing designs fit agentic serving--with LLM requests issued by agents instead of humans. This shifts the workload in two ways: (1) agents act only on complete responses, making the cluster's tokens per second (TPS) the primary goal and relaxing--not eliminating--per-token latency requirements; and (2) requests share much of their KV\$-reuse exceeds 80% of request tokens in a production trace from BAILIAN, versus 54-62% in chat. This paper first contributes a systematic study of request scheduling for agents on two real
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
- Linked via arxiv authorJiahao Wang →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorKaizhan Lin →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorKaixi Zhang →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorJinbo Han →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorXingda Wei →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorSijie Shen →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorChenguang Fang →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorWenyuan Yu →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorYirong Cheng →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Linked via arxiv authorHaibo Chen →
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling