Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation
Large audio-language models (LALMs) are increasingly used as automatic judges for speech evaluation. However, high agreement with human ratings does not guarantee that their verdicts are grounded in the audio. A judge may instead rely on specialist labels or reference data supplied by the evaluation protocol itself, taking a shortcut in place of listening to the audio. In this paper, we audit such protocol-level ``shortcuts'' in LALM judges across three common deployment protocols: feature-blueprint judging, where the audio is replaced by a structured text description of acoustic features, ref
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 55%Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor →
- PossiblePossibly related (embedding) · 48%Introducing Real World VoiceEQ: Measuring the human quality of voice AI →
- FuzzySimilar title/name (fuzzy) · 87%huggingface/speech-to-speech →
“Fuzzy title match (0.94): “Auditing Protocol-Level Shortcuts in Large Audio Language Mo” ≈ “huggingface/speech-to-speech””
- FuzzyOverlapping authors or contributors · 62%zhayujie/CowAgent →
“Shared author/contributor keys: chan”
- LinkedLinked via arxiv author · 85%Joonyong Park →
“Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation”
- LinkedLinked via arxiv author · 85%David M. Chan →
“Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation”
- LinkedLinked via arxiv author · 85%Yuki Saito →
“Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation”
- LinkedLinked via arxiv author · 85%Hiroshi Saruwatari →
“Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation”