Understanding In-context Learning of Addition via Activation Subspaces
Researchers introduced a method to localize few-shot learning in transformer models to a few attention heads, reducing the dimensionality of the mechanism underlying language model tasks to low-dimensional subspaces. This allows for a deeper understanding of the fine-grained computational structures in language models.
Save an API key to vote.