Why $\beta_1 = \beta_2$ Is Dynamically Special in Adam
Researchers investigated the role of momentum parameters in Adam, a widely used optimizer for large-scale training. They found that tying the two momentum parameters (β_1 = β_2) can preserve performance and identified a mechanism behind this phenomenon. This study contributes to the understanding of Adam's behavior and may inform the development of more efficient optimizers.
Save an API key to vote.