2021 IEEE International Conference on Image Processing
19-22 September 2021 • Anchorage, Alaska, USA
Imaging Without Borders
| Paper ID | MLR-APPL-IP-4.12 | ||
| Paper Title | WEIGHT REPARAMETRIZATION FOR BUDGET-AWARE NETWORK PRUNING | ||
| Authors | Robin Dupont, Hichem Sahbi, Sorbonne université, France; Guillaume Michel, Netatmo, France | ||
| Session | MLR-APPL-IP-4: Machine learning for image processing 4 | ||
| Location | Area D | ||
| Session Time: | Tuesday, 21 September, 13:30 - 15:00 | ||
| Presentation Time: | Tuesday, 21 September, 13:30 - 15:00 | ||
| Presentation | Poster | ||
| Topic | Applications of Machine Learning: Machine learning for image processing | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Pruning seeks to design lightweight architectures by removing redundant weights in overparameterized networks. Most of the existing techniques first remove structured sub-networks (filters, channels,...) and then fine-tune the resulting networks to maintain a high accuracy. However, removing a whole structure is a strong topological prior and recovering the accuracy, with fine-tuning, is highly cumbersome. In this paper, we introduce an "end-to-end" lightweight network design that achieves training and pruning simultaneously without fine-tuning. The design principle of our method relies on reparametrization that learns not only the weights but also the topological structure of the lightweight sub-network. This reparametrization acts as a prior (or regularizer) that defines pruning masks implicitly from the weights of the underlying network, without increasing the number of training parameters. Sparsity is induced with a budget loss that provides an accurate pruning. EExtensive experiments conducted on the CIFAR10 and the TinyImageNet datasets, using standard architectures (namely Conv4, VGG19 and ResNet18), show compelling results without fine-tuning. | ||