Symmetry Explains Most SIREN Weight-Space Gaps
A study of roughly 1.8 million fitted SIRENs finds that applying exact function-preserving symmetries reproduces 79.1 of the 80.4 accuracy points separating shared- from random-initialization models. The results show symmetry is sufficient to explain the degradation, while cautioning that this does not prove naturally occurring symmetry causes the entire gap.



