Two Axes of LLM Abstention: Answer Correctness and Question Answerability
A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise. The usual recipe thresholds a single confidence score, which cannot tell these apart. Across five instruction-tuned models from three families (2B to 14B), we find they are separate axes. Ordinary answer-confidence tracks whether an answer is right but is nearly blind to whether the question is answerable; a linear probe on hidden states does the reverse. The blind spot does not shrink with scale. It is worst on natural
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 51%The LLM "know-say gap" looks like a routing problem: you can read a model's confidence from hidden states [P] →
- PossiblePossibly related (embedding) · 50%Evaluate a model properly →
- PossiblePossibly related (embedding) · 50%LLMs know when they are wrong. I made a fix relating to Anthropic's new "global workspace" paper [R] →
- PossiblePossibly related (embedding) · 49%Competence Gate: gating tool-use on a small model's internal confidence signal instead of its verbalised one — Qwen3.5-4B, open weights [P] →
- PossiblePossibly related (embedding) · 48%Retrace-1.5B →
- LinkedLinked via arxiv author · 85%Benedikt J. Wagner →
“Two Axes of LLM Abstention: Answer Correctness and Question Answerability”