Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness

Hyun Chul Kim, Jong-Hwan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Recently, an explicit control of weight sparsity level between the layers in the deep neural network has been proposed and gainfully been utilized to resting-state fMRI (rfMRI) data. However, the reliability of the weight sparsity control scheme via the percentage of non-zero weights (PNZ) was not systematically evaluated in term of the convergence property of the sparsity levels across various scenarios of parameter changes (i.e. learning rates and initial weights). Thus, the primary aim of this study is to systematically evaluate the reliability of the PNZ based sparsity control scheme. In addition, the Hoyer's sparseness (HSP) based on the ratio of L1-and L2-norms was adopted as an alternative option to measure the weight sparsity level. To this end, the whole-brain functional connectivity of the rfMRI data from the Human Connectome Project was used as input of the autoencoder (AE) with the sparsity control scheme via either the PNZ or HSP. Then, the convergence to reach a target sparsity level and converged sparsity levels from the PNZ and HSP based schemes were compared. The presented methods and findings will benefit the training of the (stacked-) AE and/or deep neural network with the weight sparsity control scheme to ease the curse-of-dimensionality issue of very highdimensional neuroimaging data with limited available samples.

Original languageEnglish
Title of host publicationPRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781467365307
DOIs
Publication statusPublished - 2016 Aug 24
Event6th International Workshop on Pattern Recognition in Neuroimaging, PRNI 2016 - Trento, Italy
Duration: 2016 Jun 222016 Jun 24

Other

Other6th International Workshop on Pattern Recognition in Neuroimaging, PRNI 2016
CountryItaly
CityTrento
Period16/6/2216/6/24

Keywords

  • Deep neural network
  • functional connectivity
  • functional magnetic resonance imaging
  • Hoyer's sparseness
  • Human Connectome Project
  • non-zero ratio
  • Weight sparsity

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Biomedical Engineering

Fingerprint Dive into the research topics of 'Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness'. Together they form a unique fingerprint.

  • Cite this

    Kim, H. C., & Lee, J-H. (2016). Evaluation of weight sparsity control during autoencoder training of resting-state fMRI using non-zero ratio and hoyer's sparseness. In PRNI 2016 - 6th International Workshop on Pattern Recognition in Neuroimaging [7552356] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PRNI.2016.7552356