A new parameter pruning scheme for neural architectures is introduced, derived from the Fisher information metric. This method determines the optimal pruning level by computing the geodesic distance in the model space. The approach outperforms traditional magnitude pruning and local Fisher information pruning in various architectures and datasets, offering a state-of-the-art pruning methodology and a mathematically-motivated justification.