Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data

Hyun Chul Kim, Jong-Hwan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

The paper presented a systematic evaluation of the weight sparsity regularization schemes for the deep neural networks applied to the whole brain resting-state functional magnetic resonance imaging data. The weight sparsity regularization was deployed between the visible and hidden layers of the Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM), in which the L0-norm based non-zero value ratio and L1-/L2-norm based Hoyer's sparseness were used to define the weight sparsity. Also, the weight sparsity regularization schemes between the two consecutive layers (i.e. layer-wise) and between the layer and the node in the subsequent layer (i.e. node-wise) were compared in terms of the convergence property. Finally, the reproducibility of 10 sets of weight features extracted from the GB-RBMs trained using 10 sets of random initial weights was evaluated.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6150-6154
Number of pages5
ISBN (Electronic)9781509041176
DOIs
Publication statusPublished - 2017 Jun 16
Event2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - New Orleans, United States
Duration: 2017 Mar 52017 Mar 9

Other

Other2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017
CountryUnited States
CityNew Orleans
Period17/3/517/3/9

Keywords

  • Deep neural network
  • Gaussian-Bernoulli restricted Boltzmann machine
  • Hoyer's sparseness
  • Human Connectome Project
  • weight sparsity

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data'. Together they form a unique fingerprint.

  • Cite this

    Kim, H. C., & Lee, J-H. (2017). Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data. In 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings (pp. 6150-6154). [7953338] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2017.7953338