We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Semi-parametric training of autoencoders with Gaussian kernel smoothed topology learning neural networks.
- Authors
Xiang, Zhiyang; Deng, Changshou; Xiang, Xueting; Yu, Mali; Xiong, Jing
- Abstract
Autoencoders are essential for training multi-hidden layer neural networks. Parametric autoencoder trainings often require user selections of hidden neuron numbers and kernel types. In this paper, a semi-parametric autoencoder training method based on self-organized learning and incremental learning is proposed. The cost function is constructed incrementally by nonparametric learning, and the model parameter is trained by parametric learning. First, a topology learning neural network such as growing neural gas or self-organizing incremental neural network is trained to obtain a discrete representation of the training data. Second, the correlations between different dimensions are modeled as a joint distribution by the neural network representation and kernel smoothers. Finally, the loss function is defined to be the regression prediction errors with each dimension as a response variable in density regression. The parameter of kernels is selected by gradient descent which minimizes the reconstruction error on a data subset. The proposed architecture has the advantage of high training space efficiency because of incremental training, and the advantage of automated selection of hidden neuron numbers. Experiments are carried out on 4 UCI datasets and an image interpolation task. Results show that the proposed methods outperform the perceptron architecture autoencoders and the restricted Boltzmann machine in the task of nonlinear feature learning.
- Subjects
MACHINE learning; BOLTZMANN machine; TOPOLOGY; COST functions; FORECASTING; ENDOSPERM
- Publication
Neural Computing & Applications, 2020, Vol 32, Issue 9, p4933
- ISSN
0941-0643
- Publication type
Article
- DOI
10.1007/s00521-018-3897-z