Understanding In-context Learning of Addition via Activation Subspaces

Researchers introduced a method to localize few-shot learning in transformer models to a few attention heads, reducing the dimensionality of the mechanism underlying language model tasks to low-dimensional subspaces. This allows for a deeper understanding of the fine-grained computational structures in language models.

RSS Score 0 9/21/2026, 4:00:00 AM Original Source
Save an API key to vote.