AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning
Omni-modal video captioning is not merely combining visual captioning with audio transcription: a useful caption must describe how visual actions, speech, music, and sound effects co-evolve. Existing large multimodal models often fail at this relational step, treating audio and visual streams as loosely coupled observations, relying on automatic speech recognition, and under-specifying non-speech sounds and their links to visual events. We present AVSCap, a framework for audio-visual captioning centered on explicit cross-modal event binding. First, we construct AVSCap-130K, a tri-modal trainin
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 57%VioletVision-3B →
- PossiblePossibly related (embedding) · 51%modelscope/FunClip →
- PossiblePossibly related (embedding) · 49%Atomic-man007/Awesome_Multimodel_LLM →
- PossiblePossibly related (embedding) · 48%linyqh/NarratoAI →
- LinkedLinked via arxiv author · 85%Yanghai Wang →
“AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning”
- LinkedLinked via arxiv author · 85%Jiahao Wang →
“AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning”
- LinkedLinked via arxiv author · 85%Jiafu Tang →
“AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning”
- LinkedLinked via arxiv author · 85%Yuanxing Zhang →
“AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning”