Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs
Visual encoders construct a representation of the image input for Vision-Language models. How much conceptual, as opposed to immediately visible, information does this representation contain? We use canonical color as a controlled test case to ask whether vision encoders make canonical-color information linearly accessible, even when color is removed from the input image. We construct a dataset of objects with canonical colors, and probe vision encoders for both color and object identity using color and grayscale images. We find that canonical color remains decodable from grayscale images, and
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 59%VioletVision-3B →
“Fuzzy title match (0.73): “Canonical Color as a Lens into Concept Decodability in Visio” ≈ “VioletVision-3B””
- FuzzySimilar title/name (fuzzy) · 84%pytorch/vision →
“Fuzzy title match (0.92): “Canonical Color as a Lens into Concept Decodability in Visio” ≈ “pytorch/vision””
- LinkedLinked via arxiv author · 85%Xiaofu Chen →
“Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs”
- LinkedLinked via arxiv author · 85%Stella Frank →
“Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs”
- LinkedLinked via arxiv author · 85%Yova Kementchedjhieva →
“Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs”