A new method for accelerating dense Large Language Models (LLMs) using L0-regularized Mixture-of-Experts (MoE) approach is proposed. This method achieves a significant speedup of up to 2.5x without compromising performance, making it a promising solution for efficient LLM inference.

A new method called RBS-Attention is proposed to improve the efficiency of long-context large language models by reducing the cost of prefill. It uses two complementary selection branches to identify relevant tokens and achieves significant speedup and accuracy improvements.

Researchers from the Multiverse Computing and the Center for Artificial Intelligence (CAI) at the Polytechnic University of Catalonia have found a new approach to pruning large language models (LLMs) by framing it as an Ising optimization problem. This method aims to remove redundant blocks in LLMs to improve efficiency and reduce computational resources. The approach leverages insights from physics to optimize the pruning process, leading to significant reductions in the number of parameters needed to achieve a certain level of accuracy.

Researchers propose a new diffusion model sampling technique called speculative draft trees, which accelerates model generation by drafting candidate states and correcting them under a coupling that preserves the target distribution. This approach can reduce the number of expensive target evaluations, with experiments showing up to 8.3% acceleration over a baseline method.

AutoTuneBench is a new measurement protocol for trustworthy agent auto-tuning of LLM serving engines. It addresses four failure modes in current measurement methods, including strawman baselines, non-transferable absolute times, saturated tasks, and infrastructure defects. The protocol is frozen as code, uses test-enforced provenance, and includes a database-level validator, anti-cheat checks, and comparisons anchored to externally published results.

Researchers propose SKIP, a self-knowledge-guided step-wise preference learning framework for concise reasoning in large language models. This framework adjusts the model's output style and uses a knowledge probing mechanism to guide the model to output an answer at each reasoning step, improving compression while minimizing performance degradation.

A new parameter pruning scheme for neural architectures is introduced, derived from the Fisher information metric. This method determines the optimal pruning level by computing the geodesic distance in the model space. The approach outperforms traditional magnitude pruning and local Fisher information pruning in various architectures and datasets, offering a state-of-the-art pruning methodology and a mathematically-motivated justification.