Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
Vision-Language Models (VLMs) are increasingly utilized as the conditioning backbone for diffusion-based image editing due to their remarkable multimodal reasoning capabilities. While standalone VLMs demonstrate strong localization capabilities, editing pipelines frequently struggle to maintain this accuracy, particularly in complex, multi-entity scenes. In this work, we investigate this performance gap, hypothesizing that it stems from treating the VLM as a condition encoder. In this role, the model is restricted to a single forward pass, preventing the autoregressive generation process for w
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 51%VCG-team/DiffSegmenter →
- LinkedLinked via arxiv author · 85%Yoav Baron →
“Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders”
- LinkedLinked via arxiv author · 85%Sara Dorfman →
“Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders”
- LinkedLinked via arxiv author · 85%Roni Paiss →
“Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders”
- LinkedLinked via arxiv author · 85%Daniel Cohen-Or →
“Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders”
- LinkedLinked via arxiv author · 85%Or Patashnik →
“Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders”