We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation.
- Authors
Niu, Hao; Yi, Zhang; He, Tao
- Abstract
Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized various image recognition tasks. In this paper, we propose a novel architecture called bidirectional FNN (BiFNN), which utilizes skip connections to aggregate features between its forward and backward paths. The BiFNN accepts any FNN as a plugin that can incorporate any general FNN model into its forward path, introducing only a few additional parameters in the cross-path connections. The backward path is implemented as a nonparameter layer, utilizing a discretized form of the neural memory Ordinary Differential Equation (nmODE), which is named ϵ -net. We provide a proof of convergence for the ϵ -net and evaluate its initial value problem. Our proposed architecture is evaluated on diverse image recognition datasets, including Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet. The results demonstrate that BiFNNs offer significant improvements compared to embedded models such as ConvMixer, ResNet, ResNeXt, and Vision Transformer. Furthermore, BiFNNs can be fine-tuned to achieve comparable performance with embedded models on Tiny-ImageNet and ImageNet-1K datasets by loading the same pretrained parameters.
- Subjects
ORDINARY differential equations; TRANSFORMER models; FEEDFORWARD neural networks; IMAGE recognition (Computer vision); INITIAL value problems
- Publication
International Journal of Neural Systems, 2024, Vol 34, Issue 4, p1
- ISSN
0129-0657
- Publication type
Article
- DOI
10.1142/S0129065724500151