L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
A new routing framework, Low-rank and Lipschitz-controlled Routing (L2R), is proposed for Mixture-of-Experts (MoE) models. L2R reshapes the routing space and scoring geometry, improving routing discriminability and stable expert specialization. This matters to people building AI agents as it could lead to improved model performance and more efficient expert expressiveness.
Save an API key to vote.