Higher-order pruning of experts in mixture-of-experts language models
Researchers proposed HOPE (Higher-Order Pruning of Experts), a method for reducing the parameter count in mixture-of-experts language models by preserving cooperative expert structure. HOPE outperforms existing methods, especially at high pruning rates and on challenging tasks like agentic coding.
Save an API key to vote.