Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data

Hyun Chul Kim, Jong-Hwan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

The paper presented a systematic evaluation of the weight sparsity regularization schemes for the deep neural networks applied to the whole brain resting-state functional magnetic resonance imaging data. The weight sparsity regularization was deployed between the visible and hidden layers of the Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM), in which the L0-norm based non-zero value ratio and L1-/L2-norm based Hoyer's sparseness were used to define the weight sparsity. Also, the weight sparsity regularization schemes between the two consecutive layers (i.e. layer-wise) and between the layer and the node in the subsequent layer (i.e. node-wise) were compared in terms of the convergence property. Finally, the reproducibility of 10 sets of weight features extracted from the GB-RBMs trained using 10 sets of random initial weights was evaluated.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6150-6154
Number of pages5
ISBN (Electronic)9781509041176
DOIs
Publication statusPublished - 2017 Jun 16
Event2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - New Orleans, United States
Duration: 2017 Mar 52017 Mar 9

Other

Other2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017
CountryUnited States
CityNew Orleans
Period17/3/517/3/9

Fingerprint

Functional neuroimaging
Brain
Deep neural networks
Magnetic Resonance Imaging

Keywords

  • Deep neural network
  • Gaussian-Bernoulli restricted Boltzmann machine
  • Hoyer's sparseness
  • Human Connectome Project
  • weight sparsity

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Kim, H. C., & Lee, J-H. (2017). Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data. In 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings (pp. 6150-6154). [7953338] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2017.7953338

Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data. / Kim, Hyun Chul; Lee, Jong-Hwan.

2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 6150-6154 7953338.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, HC & Lee, J-H 2017, Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data. in 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings., 7953338, Institute of Electrical and Electronics Engineers Inc., pp. 6150-6154, 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017, New Orleans, United States, 17/3/5. https://doi.org/10.1109/ICASSP.2017.7953338
Kim HC, Lee J-H. Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data. In 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. p. 6150-6154. 7953338 https://doi.org/10.1109/ICASSP.2017.7953338
Kim, Hyun Chul ; Lee, Jong-Hwan. / Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data. 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 6150-6154
@inproceedings{db90adf6eb414082a244bc221e050893,
title = "Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data",
abstract = "The paper presented a systematic evaluation of the weight sparsity regularization schemes for the deep neural networks applied to the whole brain resting-state functional magnetic resonance imaging data. The weight sparsity regularization was deployed between the visible and hidden layers of the Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM), in which the L0-norm based non-zero value ratio and L1-/L2-norm based Hoyer's sparseness were used to define the weight sparsity. Also, the weight sparsity regularization schemes between the two consecutive layers (i.e. layer-wise) and between the layer and the node in the subsequent layer (i.e. node-wise) were compared in terms of the convergence property. Finally, the reproducibility of 10 sets of weight features extracted from the GB-RBMs trained using 10 sets of random initial weights was evaluated.",
keywords = "Deep neural network, Gaussian-Bernoulli restricted Boltzmann machine, Hoyer's sparseness, Human Connectome Project, weight sparsity",
author = "Kim, {Hyun Chul} and Jong-Hwan Lee",
year = "2017",
month = "6",
day = "16",
doi = "10.1109/ICASSP.2017.7953338",
language = "English",
pages = "6150--6154",
booktitle = "2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Evaluation of weight sparsity regularizion schemes of deep neural networks applied to functional neuroimaging data

AU - Kim, Hyun Chul

AU - Lee, Jong-Hwan

PY - 2017/6/16

Y1 - 2017/6/16

N2 - The paper presented a systematic evaluation of the weight sparsity regularization schemes for the deep neural networks applied to the whole brain resting-state functional magnetic resonance imaging data. The weight sparsity regularization was deployed between the visible and hidden layers of the Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM), in which the L0-norm based non-zero value ratio and L1-/L2-norm based Hoyer's sparseness were used to define the weight sparsity. Also, the weight sparsity regularization schemes between the two consecutive layers (i.e. layer-wise) and between the layer and the node in the subsequent layer (i.e. node-wise) were compared in terms of the convergence property. Finally, the reproducibility of 10 sets of weight features extracted from the GB-RBMs trained using 10 sets of random initial weights was evaluated.

AB - The paper presented a systematic evaluation of the weight sparsity regularization schemes for the deep neural networks applied to the whole brain resting-state functional magnetic resonance imaging data. The weight sparsity regularization was deployed between the visible and hidden layers of the Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM), in which the L0-norm based non-zero value ratio and L1-/L2-norm based Hoyer's sparseness were used to define the weight sparsity. Also, the weight sparsity regularization schemes between the two consecutive layers (i.e. layer-wise) and between the layer and the node in the subsequent layer (i.e. node-wise) were compared in terms of the convergence property. Finally, the reproducibility of 10 sets of weight features extracted from the GB-RBMs trained using 10 sets of random initial weights was evaluated.

KW - Deep neural network

KW - Gaussian-Bernoulli restricted Boltzmann machine

KW - Hoyer's sparseness

KW - Human Connectome Project

KW - weight sparsity

UR - http://www.scopus.com/inward/record.url?scp=85023779810&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85023779810&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2017.7953338

DO - 10.1109/ICASSP.2017.7953338

M3 - Conference contribution

AN - SCOPUS:85023779810

SP - 6150

EP - 6154

BT - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -