newsReddit r/MachineLearningTrust 52 · CommunityPublished 1mo agoLive · 1mo ago
Competence Gate: gating tool-use on a small model's internal confidence signal instead of its verbalised one — Qwen3.5-4B, open weights [P]
I made a 10MB LoRA adapter for Qwen3.5-4B plus a small orchestration layer. It decides, per query, whether to answer directly, search the web, or retrieve from your own local documents and it refuses to make things up when it can't verify an answer. It runs locally (Apple Silicon / MLX, with a GGUF build for llama.cpp/Ollama). Basically small instruct models are poor at telling users how confident they really are. They can't verbalise it and tend t
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 51%Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA →
- PossiblePossibly related (embedding) · 50%Fine-tune a small model on your own data →
- PossiblePossibly related (embedding) · 47%Evil Spectra: How Optimisers can Amplify or Suppress Emergent Misalignment →
- PossiblePossibly related (embedding) · 46%QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents →
- PossiblePossibly related (embedding) · 45%Evaluate a model properly →
- PossiblePossibly related (embedding) · 49%Two Axes of LLM Abstention: Answer Correctness and Question Answerability →
- PossiblePossibly related (embedding) · 46%LLM Judges Can Be Too Generous When There Is No Reference Answer →
- PossiblePossibly related (embedding) · 50%Phantom Gains: Auditing Self-Improvement Against a Measured Null →
Covers
paperCan LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QAtutorialFine-tune a small model on your own datapaperEvil Spectra: How Optimisers can Amplify or Suppress Emergent MisalignmentpaperQVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM AgentstutorialEvaluate a model properly
Covers (incoming)
Related across the graph
tutorialFine-tune a small model on your own datapaperEvil Spectra: How Optimisers can Amplify or Suppress Emergent MisalignmentpaperLLM Judges Can Be Too Generous When There Is No Reference AnswerpaperQVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM AgentspaperTwo Axes of LLM Abstention: Answer Correctness and Question AnswerabilitypaperPhantom Gains: Auditing Self-Improvement Against a Measured NullpaperCan LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QAtutorialEvaluate a model properly