Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D
While large language models (LLMs) can solve advanced reasoning problems in seconds, we show that even frontier models fail to perform a much simpler operation: exactly copying an input string that lies well within their context windows. We attribute this failure to positional encodings in Transformer architectures, whose inductive bias favors copying through a shortcut based on matching local contexts rather than carefully locating the corresponding input positions. To address this issue, we introduce 2D-RoPE, which organizes text into a 2D grid rather than a 1D sequence and assigns each toke
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 51%Transformer →
- LinkedLinked via arxiv author · 85%Haodong Wen →
“Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D”
- LinkedLinked via arxiv author · 85%Yiran Zhang →
“Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D”
- LinkedLinked via arxiv author · 85%Yingfa Chen →
“Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D”
- LinkedLinked via arxiv author · 85%Kaifeng Lyu →
“Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D”