Safe semi-supervised learning using a bayesian neural network

Jinsoo Bae, Minjung Lee, Seoung Bum Kim

Research output: Contribution to journalArticlepeer-review

Abstract

Semi-supervised learning attempts to use a large set of unlabeled data to increase the prediction accuracy of machine learning models when the amount of labeled data is limited. However, in realistic cases, unlabeled data may worsen performance because they contain out-of-distribution (OOD) data that differ from the labeled data. To address this issue, safe semi-supervised deep learning has recently been presented. This study suggests a new safe semi-supervised algorithm that uses an uncertainty-aware Bayesian neural network. Our proposed method, safe uncertainty-based consistency training (SafeUC), uses Bayesian uncertainty to minimize the harmful effects caused by unlabeled OOD examples. The proposed method improves the model's generalization performance by regularizing the network for consistency against uncertain noise. Moreover, to avoid uncertain prediction results, the proposed method includes a practical inference tip based on a well-calibrated uncertainty. The effectiveness of the proposed method is demonstrated in the experimental results on CIFAR-10 and SVHN by showing that it achieved state-of-the-art performance for all semi-supervised learning tasks with OOD data presence rates.

Original languageEnglish
Pages (from-to)453-464
Number of pages12
JournalInformation Sciences
Volume612
DOIs
Publication statusPublished - 2022 Oct

Keywords

  • Bayesian neural network
  • Consistency regularization
  • Out-of-distribution
  • Safe semi-supervised deep learning
  • Uncertain noise
  • Uncertainty

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Software
  • Control and Systems Engineering
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Safe semi-supervised learning using a bayesian neural network'. Together they form a unique fingerprint.

Cite this