Sparse Competition during Training For the Emergence of Specialized Modules
Modularity in deep neural networks has been proposed as a means of improving both interpretability and training by promoting disentangled representations and reducing redundancy. In this work, we study the emergence of modular structure through competition dynamics between groups of neurons during training. We introduce a method that (i) maintains near-baseline accuracy, (ii) induces usage-based modularity by sparsely routing inputs to neuron groups, and (iii) encourages specialization of these modules, such that their activations are correlated with input classes. We evaluate the proposed app
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 49%A unifying framework from neural superposition to sparse interpretable codes →
- PossiblePossibly related (embedding) · 48%A unifying framework from neural superposition to sparse interpretable codes - Nature →
- LinkedLinked via arxiv author · 85%Baptiste Rossigneux →
“Sparse Competition during Training For the Emergence of Specialized Modules”
- LinkedLinked via arxiv author · 85%Karim Haroun →
“Sparse Competition during Training For the Emergence of Specialized Modules”