We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Network Adjustment: Channel and Block Search Guided by Resource Utilization Ratio.
- Authors
Chen, Zhengsu; Xie, Lingxi; Niu, Jianwei; Liu, Xuefeng; Wei, Longhui; Tian, Qi
- Abstract
It is an important problem to design resource-efficient neural architectures. One solution is adjusting the number of channels in each layer and the number of blocks in each network stage. This paper presents a novel framework named network adjustment which considers accuracy as a function of the computational resource (e.g., FLOPs or parameters), so that architecture design becomes an optimization problem and can be solved with the gradient-based optimization method. The gradient is defined as the resource utilization ratio (RUR) of each changeable module (layer or block) in a network and is accurate only in a small neighborhood of the current status. Therefore, we estimate it using Dropout, a probabilistic operation, and optimize the network architecture iteratively. The computational overhead of the entire process is comparable to that of re-training the final model from scratch. We investigate two versions of RUR where the resource usage is measured by FLOPs and latency. Experiments on standard image classification datasets and a few base networks including ResNet and EfficientNet demonstrate the effectiveness of our approach, which consistently outperforms the pruning-based counterparts.
- Subjects
ARCHITECTURAL design; NEIGHBORHOODS
- Publication
International Journal of Computer Vision, 2022, Vol 130, Issue 3, p820
- ISSN
0920-5691
- Publication type
Article
- DOI
10.1007/s11263-021-01566-5