Why $\beta_1 = \beta_2$ Is Dynamically Special in Adam

Researchers investigated the role of momentum parameters in Adam, a widely used optimizer for large-scale training. They found that tying the two momentum parameters (β_1 = β_2) can preserve performance and identified a mechanism behind this phenomenon. This study contributes to the understanding of Adam's behavior and may inform the development of more efficient optimizers.

RSS Score 0 9/18/2026, 4:00:00 AM Original Source
Save an API key to vote.