Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment

Joongho Jo; Jongsun Park

doi:10.1109/ISOCC50952.2020.9332956

Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment

Joongho Jo, Jongsun Park

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

As the convolutional neural network becomes complex and datasets become huge, a lot of time is spent training the network. In this paper, we propose to mitigate this phenomenon with a mini-batch skipping strategy based on an arithmetic mean of confidence score of images. By skipping the unimportant mini-batch on the training phase, the mini-batch skipping provides saving a lot of time on backpropagation and weight update. We empirically demonstrate the effectiveness of our method with Resnet-18, Resnet-50, and mobilenet-v2 on Cifar-10 and Cifar-100. For Res-net-50, mini-batch skipping gives a 1.39x speedup in training operation without significant accuracy drop.

Original language	English
Title of host publication	Proceedings - International SoC Design Conference, ISOCC 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	129-130
Number of pages	2
ISBN (Electronic)	9781728183312
DOIs	https://doi.org/10.1109/ISOCC50952.2020.9332956
Publication status	Published - 2020 Oct 21
Event	17th International System-on-Chip Design Conference, ISOCC 2020 - Yeosu, Korea, Republic of Duration: 2020 Oct 21 → 2020 Oct 24

Publication series

Name	Proceedings - International SoC Design Conference, ISOCC 2020

Conference

Conference	17th International System-on-Chip Design Conference, ISOCC 2020
Country/Territory	Korea, Republic of
City	Yeosu
Period	20/10/21 → 20/10/24

Keywords

Convolutional Neural Network (CNN)
mini-batch skipping

ASJC Scopus subject areas

Energy Engineering and Power Technology
Electrical and Electronic Engineering
Instrumentation
Artificial Intelligence
Hardware and Architecture

Access to Document

10.1109/ISOCC50952.2020.9332956

Cite this

Jo, J., & Park, J. (2020). Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment. In Proceedings - International SoC Design Conference, ISOCC 2020 (pp. 129-130). [9332956] (Proceedings - International SoC Design Conference, ISOCC 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISOCC50952.2020.9332956

Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment. / Jo, Joongho; Park, Jongsun.

Proceedings - International SoC Design Conference, ISOCC 2020. Institute of Electrical and Electronics Engineers Inc., 2020. p. 129-130 9332956 (Proceedings - International SoC Design Conference, ISOCC 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jo, J & Park, J 2020, Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment. in Proceedings - International SoC Design Conference, ISOCC 2020., 9332956, Proceedings - International SoC Design Conference, ISOCC 2020, Institute of Electrical and Electronics Engineers Inc., pp. 129-130, 17th International System-on-Chip Design Conference, ISOCC 2020, Yeosu, Korea, Republic of, 20/10/21. https://doi.org/10.1109/ISOCC50952.2020.9332956

@inproceedings{d730fe781e39495e8357085e72c12482,

title = "Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment",

abstract = "As the convolutional neural network becomes complex and datasets become huge, a lot of time is spent training the network. In this paper, we propose to mitigate this phenomenon with a mini-batch skipping strategy based on an arithmetic mean of confidence score of images. By skipping the unimportant mini-batch on the training phase, the mini-batch skipping provides saving a lot of time on backpropagation and weight update. We empirically demonstrate the effectiveness of our method with Resnet-18, Resnet-50, and mobilenet-v2 on Cifar-10 and Cifar-100. For Res-net-50, mini-batch skipping gives a 1.39x speedup in training operation without significant accuracy drop. ",

keywords = "Convolutional Neural Network (CNN), mini-batch skipping",

author = "Joongho Jo and Jongsun Park",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 17th International System-on-Chip Design Conference, ISOCC 2020 ; Conference date: 21-10-2020 Through 24-10-2020",

year = "2020",

month = oct,

day = "21",

doi = "10.1109/ISOCC50952.2020.9332956",

language = "English",

series = "Proceedings - International SoC Design Conference, ISOCC 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "129--130",

booktitle = "Proceedings - International SoC Design Conference, ISOCC 2020",

}

TY - GEN

T1 - Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment

AU - Jo, Joongho

AU - Park, Jongsun

PY - 2020/10/21

Y1 - 2020/10/21

N2 - As the convolutional neural network becomes complex and datasets become huge, a lot of time is spent training the network. In this paper, we propose to mitigate this phenomenon with a mini-batch skipping strategy based on an arithmetic mean of confidence score of images. By skipping the unimportant mini-batch on the training phase, the mini-batch skipping provides saving a lot of time on backpropagation and weight update. We empirically demonstrate the effectiveness of our method with Resnet-18, Resnet-50, and mobilenet-v2 on Cifar-10 and Cifar-100. For Res-net-50, mini-batch skipping gives a 1.39x speedup in training operation without significant accuracy drop.

AB - As the convolutional neural network becomes complex and datasets become huge, a lot of time is spent training the network. In this paper, we propose to mitigate this phenomenon with a mini-batch skipping strategy based on an arithmetic mean of confidence score of images. By skipping the unimportant mini-batch on the training phase, the mini-batch skipping provides saving a lot of time on backpropagation and weight update. We empirically demonstrate the effectiveness of our method with Resnet-18, Resnet-50, and mobilenet-v2 on Cifar-10 and Cifar-100. For Res-net-50, mini-batch skipping gives a 1.39x speedup in training operation without significant accuracy drop.

KW - Convolutional Neural Network (CNN)

KW - mini-batch skipping

UR - http://www.scopus.com/inward/record.url?scp=85100755744&partnerID=8YFLogxK

U2 - 10.1109/ISOCC50952.2020.9332956

DO - 10.1109/ISOCC50952.2020.9332956

M3 - Conference contribution

AN - SCOPUS:85100755744

T3 - Proceedings - International SoC Design Conference, ISOCC 2020

SP - 129

EP - 130

BT - Proceedings - International SoC Design Conference, ISOCC 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th International System-on-Chip Design Conference, ISOCC 2020

Y2 - 21 October 2020 through 24 October 2020

ER -

Confidence Score based Mini-batch Skipping for CNN Training on Mini-batch Training Environment

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this