Structured Layerwise Pruning via Extended-Interval Activation- and Derivative-Based Significance Measures

International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP), pp. 1–6

Abstract

Neural networks often suffer from over-parameterization, which limits their deployment on resource-constrained devices due to high memory and computational demands. Pruning can mitigate this by removing redundant components while maintaining predictive performance. In this work, we study structured, data-driven pruning strategies that assess neuron significance using either activations or derivatives. To capture neuron influence beyond sampled data, these measures are extended with interval arithmetic, enabling evaluation across the entire input domain. We employ a layerwise iterative pruning strategy to progressively remove the least significant neurons, both with and without retraining. We evaluate four significance measures on regression and classification tasks using a fully connected three-layer network, providing insights into trade-offs between model compression and predictive performance under different pruning criteria.

Authors 4

1. Mansoor Ahmad Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Aachen,Germany

2. Sher Afghan Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Aachen,Germany

3. Uwe Naumann Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Aachen,Germany

4. Ekkapot Charoenwanit

   King Mongkut's University of Technology North Bangkok

   Affiliation as printed

   King Mongkut's University of Technology North Bangkok,Bangkok,Thailand