oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions
Hyper-Connections (HC) replace the single residual stream of a Transformer with $n$ parallel ones, mixing them at every layer with a learned $n \times n$ residual matrix. Leaving that matrix unconstrained places no limit on the factor by which the mixing step rescales the residual streams, and that factor compounds across layers, which destabilizes training. Manifold-constrained Hyper-Connections (mHC) address this by restricting the matrix to the doubly stochastic matrices. That caps the factor at one, so the mixing can no longer amplify any direction, but nothing bounds it from below. We pro
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzyOverlapping authors or contributors · 62%mudler/LocalAI →
“Shared author/contributor keys: guo”
- FuzzyOverlapping authors or contributors · 62%sgl-project/sglang →
“Shared author/contributor keys: luo”
- LinkedLinked via arxiv author · 85%Haoqiang Guo →
“oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions”
- LinkedLinked via arxiv author · 85%Xuyi Chen →
“oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions”
- LinkedLinked via arxiv author · 85%Bo Ke →
“oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions”
- LinkedLinked via arxiv author · 85%Yishu Lei →
“oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions”
- LinkedLinked via arxiv author · 85%Ziyang Xu →
“oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions”
- LinkedLinked via arxiv author · 85%Shikun Feng →
“oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions”