An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation
While Large Multimodal Models excel in comprehension, high-throughput inference engines lack native support for multimodal generation. This is severe in Speech Language Models, where generating multi-layered audio tokens via decoupled AR+NAR or synchronous Multi-Token Prediction (MTP) with delay-pattern interleaving conflicts with standard single-stream loops. We present a vLLM-based inference pipeline for unified speech understanding and generation. We extend autoregressive decoding to natively execute delay-pattern de-interleaving and coordinated multi-stream sampling, integrating an on-GPU
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 53%thu-pacman/chitu →
- PossiblePossibly related (embedding) · 48%vllm-project/vllm →
- PossiblePossibly related (embedding) · 48%jaswon/osu-dreamer →
- PossiblePossibly related (embedding) · 46%sgl-project/sglang →
- PossiblePossibly related (embedding) · 45%Whisper-Lite →
- LinkedLinked via arxiv author · 85%Haoran Wang →
“An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation”
- LinkedLinked via arxiv author · 85%Jinchuan Tian →
“An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation”
- LinkedLinked via arxiv author · 85%Siddhant Arora →
“An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation”