Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness

Hyun Chul Kim, Jong-Hwan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Recently, an explicit control of weight sparsity level between the layers in the deep neural network has been proposed and gainfully been utilized to resting-state fMRI (rfMRI) data. However, the reliability of the weight sparsity control scheme via the percentage of non-zero weights (PNZ) was not systematically evaluated in term of the convergence property of the sparsity levels across various scenarios of parameter changes (i.e. learning rates and initial weights). Thus, the primary aim of this study is to systematically evaluate the reliability of the PNZ based sparsity control scheme. In addition, the Hoyer's sparseness (HSP) based on the ratio of L1-and L2-norms was adopted as an alternative option to measure the weight sparsity level. To this end, the whole-brain functional connectivity of the rfMRI data from the Human Connectome Project was used as input of the autoencoder (AE) with the sparsity control scheme via either the PNZ or HSP. Then, the convergence to reach a target sparsity level and converged sparsity levels from the PNZ and HSP based schemes were compared. The presented methods and findings will benefit the training of the (stacked-) AE and/or deep neural network with the weight sparsity control scheme to ease the curse-of-dimensionality issue of very highdimensional neuroimaging data with limited available samples.

Original languageEnglish
Title of host publicationPRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781467365307
DOIs
Publication statusPublished - 2016 Aug 24
Event6th International Workshop on Pattern Recognition in Neuroimaging, PRNI 2016 - Trento, Italy
Duration: 2016 Jun 222016 Jun 24

Other

Other6th International Workshop on Pattern Recognition in Neuroimaging, PRNI 2016
CountryItaly
CityTrento
Period16/6/2216/6/24

Fingerprint

Weight control
Neuroimaging
Brain
Magnetic Resonance Imaging
Deep neural networks

Keywords

  • Deep neural network
  • functional connectivity
  • functional magnetic resonance imaging
  • Hoyer's sparseness
  • Human Connectome Project
  • non-zero ratio
  • Weight sparsity

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Biomedical Engineering

Cite this

Kim, H. C., & Lee, J-H. (2016). Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness. In PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging [7552356] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PRNI.2016.7552356

Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness. / Kim, Hyun Chul; Lee, Jong-Hwan.

PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging. Institute of Electrical and Electronics Engineers Inc., 2016. 7552356.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, HC & Lee, J-H 2016, Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness. in PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging., 7552356, Institute of Electrical and Electronics Engineers Inc., 6th International Workshop on Pattern Recognition in Neuroimaging, PRNI 2016, Trento, Italy, 16/6/22. https://doi.org/10.1109/PRNI.2016.7552356
Kim HC, Lee J-H. Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness. In PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging. Institute of Electrical and Electronics Engineers Inc. 2016. 7552356 https://doi.org/10.1109/PRNI.2016.7552356
Kim, Hyun Chul ; Lee, Jong-Hwan. / Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness. PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging. Institute of Electrical and Electronics Engineers Inc., 2016.
@inproceedings{3d4d3fb08c794e2481052fe0c0500d57,
title = "Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness",
abstract = "Recently, an explicit control of weight sparsity level between the layers in the deep neural network has been proposed and gainfully been utilized to resting-state fMRI (rfMRI) data. However, the reliability of the weight sparsity control scheme via the percentage of non-zero weights (PNZ) was not systematically evaluated in term of the convergence property of the sparsity levels across various scenarios of parameter changes (i.e. learning rates and initial weights). Thus, the primary aim of this study is to systematically evaluate the reliability of the PNZ based sparsity control scheme. In addition, the Hoyer's sparseness (HSP) based on the ratio of L1-and L2-norms was adopted as an alternative option to measure the weight sparsity level. To this end, the whole-brain functional connectivity of the rfMRI data from the Human Connectome Project was used as input of the autoencoder (AE) with the sparsity control scheme via either the PNZ or HSP. Then, the convergence to reach a target sparsity level and converged sparsity levels from the PNZ and HSP based schemes were compared. The presented methods and findings will benefit the training of the (stacked-) AE and/or deep neural network with the weight sparsity control scheme to ease the curse-of-dimensionality issue of very highdimensional neuroimaging data with limited available samples.",
keywords = "Deep neural network, functional connectivity, functional magnetic resonance imaging, Hoyer's sparseness, Human Connectome Project, non-zero ratio, Weight sparsity",
author = "Kim, {Hyun Chul} and Jong-Hwan Lee",
year = "2016",
month = "8",
day = "24",
doi = "10.1109/PRNI.2016.7552356",
language = "English",
booktitle = "PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness

AU - Kim, Hyun Chul

AU - Lee, Jong-Hwan

PY - 2016/8/24

Y1 - 2016/8/24

N2 - Recently, an explicit control of weight sparsity level between the layers in the deep neural network has been proposed and gainfully been utilized to resting-state fMRI (rfMRI) data. However, the reliability of the weight sparsity control scheme via the percentage of non-zero weights (PNZ) was not systematically evaluated in term of the convergence property of the sparsity levels across various scenarios of parameter changes (i.e. learning rates and initial weights). Thus, the primary aim of this study is to systematically evaluate the reliability of the PNZ based sparsity control scheme. In addition, the Hoyer's sparseness (HSP) based on the ratio of L1-and L2-norms was adopted as an alternative option to measure the weight sparsity level. To this end, the whole-brain functional connectivity of the rfMRI data from the Human Connectome Project was used as input of the autoencoder (AE) with the sparsity control scheme via either the PNZ or HSP. Then, the convergence to reach a target sparsity level and converged sparsity levels from the PNZ and HSP based schemes were compared. The presented methods and findings will benefit the training of the (stacked-) AE and/or deep neural network with the weight sparsity control scheme to ease the curse-of-dimensionality issue of very highdimensional neuroimaging data with limited available samples.

AB - Recently, an explicit control of weight sparsity level between the layers in the deep neural network has been proposed and gainfully been utilized to resting-state fMRI (rfMRI) data. However, the reliability of the weight sparsity control scheme via the percentage of non-zero weights (PNZ) was not systematically evaluated in term of the convergence property of the sparsity levels across various scenarios of parameter changes (i.e. learning rates and initial weights). Thus, the primary aim of this study is to systematically evaluate the reliability of the PNZ based sparsity control scheme. In addition, the Hoyer's sparseness (HSP) based on the ratio of L1-and L2-norms was adopted as an alternative option to measure the weight sparsity level. To this end, the whole-brain functional connectivity of the rfMRI data from the Human Connectome Project was used as input of the autoencoder (AE) with the sparsity control scheme via either the PNZ or HSP. Then, the convergence to reach a target sparsity level and converged sparsity levels from the PNZ and HSP based schemes were compared. The presented methods and findings will benefit the training of the (stacked-) AE and/or deep neural network with the weight sparsity control scheme to ease the curse-of-dimensionality issue of very highdimensional neuroimaging data with limited available samples.

KW - Deep neural network

KW - functional connectivity

KW - functional magnetic resonance imaging

KW - Hoyer's sparseness

KW - Human Connectome Project

KW - non-zero ratio

KW - Weight sparsity

UR - http://www.scopus.com/inward/record.url?scp=84988811099&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988811099&partnerID=8YFLogxK

U2 - 10.1109/PRNI.2016.7552356

DO - 10.1109/PRNI.2016.7552356

M3 - Conference contribution

AN - SCOPUS:84988811099

BT - PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging

PB - Institute of Electrical and Electronics Engineers Inc.

ER -