Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not robustly internalized. We study this as a source of hallucinations and frame a group of mitigation methods as \emph{knowledge-aligned SFT}: constraining SFT training targets to the base model's parametric knowledge. Under a unified setup, we compare existing generation-based and estimation-based knowledge-alignment methods and introduce two new variants: Evidence Rewrite, which verifies base-model generations using external evidence, and Recall R
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 55%Towards principled knowledge editing methods for large language model reasoning →
- PossiblePossibly related (embedding) · 55%Preparing data for supervised fine-tuning Part 1: Formatting and quality →
- LinkedLinked via arxiv author · 85%Arthur Becker →
“Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning”
- LinkedLinked via arxiv author · 85%Jakob Kemmler →
“Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning”
- LinkedLinked via arxiv author · 85%David Thulke →
“Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning”
- LinkedLinked via arxiv author · 85%Christine Schäfer →
“Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning”
- LinkedLinked via arxiv author · 85%Christian Dugast →
“Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning”
- LinkedLinked via arxiv author · 85%Hermann Ney →
“Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning”