MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
Apr 23, 2019
MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
-
Gordon, Ariel;
-
Eban, Elad;
-
Nachum, Ofir;
-
Chen, Bo;
-
Wu, Hao;
-
Yang, Tien-Ju;
-
Choi, Edward;
Abstract: We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our method is scalable to large networks, adaptable to specific resource constraints (e.g. the number of floating-point operations per inference), and capable of increasing the network's performance. When applied to standard network architectures on a wide variety of datasets, our approach discovers novel structures in each domain, obtaining higher performance while respecting the resource constraint.
https://arxiv.org/abs/1711.06798
← Back to all articles Quick Navigation: Next:[ j ] – Prev:[ k ] – List:[ l ]