Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch

Researchers introduce Transition Games to study the transition from memorization to generalization in Transformers. They find that a prospective block-0 attention bias and selected degree-two modes account for a significant portion of the gain in distributed utility. This discovery identifies grokking as spectral recoding of an existing distributed circuit rather than a module switch.

RSS Score 0 9/17/2026, 4:00:00 AM Original Source
Save an API key to vote.