Researchers introduced the mean-expansion layer to accelerate action-value learning in reinforcement learning, improving performance in deep Q-networks and implicit quantile networks by sharing values across actions within a state.