Multipath-DenseNet

A Supervised ensemble architecture of densely connected convolutional networks

Bilal Lodhi, Jaewoo Kang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Deep networks with skip-connections such as ResNets have achieved great results in recent years. DenseNet exploits the ResNet skip-connections by connecting each layer in convolution neural network to all preceding layers and achieves state-of-the-art accuracy. It is well-known that deeper networks are more efficient and easier to train than shallow or wider networks. Despite the high performance of very deep networks, they are limited in terms of vanishing gradient, diminishing forward flow, and slower training time. In this paper, we propose to combine the benefits of the depth and width of networks. We train supervised independent shallow networks on the same input in a block fashion. We use a state-of-the-art DenseNet block to increase the number of paths for gradient flow. Our proposed architecture has several advantages over other deeper networks including DenseNet; our architecture which we call Multipath-DenseNet is deeper as well as wider, reduces training time, and uses a smaller number of parameters. We evaluate our proposed architecture on the following four object recognition datasets: CIFAR-10, CIFAR-100, SVHN, and ImageNet. The evaluation results show that Multipath-DenseNet achieves significant improvement in performance over DenseNet on the benchmark datasets.

Original languageEnglish
Pages (from-to)63-72
Number of pages10
JournalInformation Sciences
Volume482
DOIs
Publication statusPublished - 2019 May 1

Fingerprint

Object recognition
Multipath
Convolution
Ensemble
Neural networks
Gradient Flow
Diminishing
Object Recognition
Architecture
High Performance
Neural Networks
Benchmark
Gradient
Path
Evaluate
Evaluation

Keywords

  • Deep-learning
  • Image classification
  • Neural network

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Cite this

Multipath-DenseNet : A Supervised ensemble architecture of densely connected convolutional networks. / Lodhi, Bilal; Kang, Jaewoo.

In: Information Sciences, Vol. 482, 01.05.2019, p. 63-72.

Research output: Contribution to journalArticle

@article{b9cc516ad91b428db73931630c0e9e2f,
title = "Multipath-DenseNet: A Supervised ensemble architecture of densely connected convolutional networks",
abstract = "Deep networks with skip-connections such as ResNets have achieved great results in recent years. DenseNet exploits the ResNet skip-connections by connecting each layer in convolution neural network to all preceding layers and achieves state-of-the-art accuracy. It is well-known that deeper networks are more efficient and easier to train than shallow or wider networks. Despite the high performance of very deep networks, they are limited in terms of vanishing gradient, diminishing forward flow, and slower training time. In this paper, we propose to combine the benefits of the depth and width of networks. We train supervised independent shallow networks on the same input in a block fashion. We use a state-of-the-art DenseNet block to increase the number of paths for gradient flow. Our proposed architecture has several advantages over other deeper networks including DenseNet; our architecture which we call Multipath-DenseNet is deeper as well as wider, reduces training time, and uses a smaller number of parameters. We evaluate our proposed architecture on the following four object recognition datasets: CIFAR-10, CIFAR-100, SVHN, and ImageNet. The evaluation results show that Multipath-DenseNet achieves significant improvement in performance over DenseNet on the benchmark datasets.",
keywords = "Deep-learning, Image classification, Neural network",
author = "Bilal Lodhi and Jaewoo Kang",
year = "2019",
month = "5",
day = "1",
doi = "10.1016/j.ins.2019.01.012",
language = "English",
volume = "482",
pages = "63--72",
journal = "Information Sciences",
issn = "0020-0255",
publisher = "Elsevier Inc.",

}

TY - JOUR

T1 - Multipath-DenseNet

T2 - A Supervised ensemble architecture of densely connected convolutional networks

AU - Lodhi, Bilal

AU - Kang, Jaewoo

PY - 2019/5/1

Y1 - 2019/5/1

N2 - Deep networks with skip-connections such as ResNets have achieved great results in recent years. DenseNet exploits the ResNet skip-connections by connecting each layer in convolution neural network to all preceding layers and achieves state-of-the-art accuracy. It is well-known that deeper networks are more efficient and easier to train than shallow or wider networks. Despite the high performance of very deep networks, they are limited in terms of vanishing gradient, diminishing forward flow, and slower training time. In this paper, we propose to combine the benefits of the depth and width of networks. We train supervised independent shallow networks on the same input in a block fashion. We use a state-of-the-art DenseNet block to increase the number of paths for gradient flow. Our proposed architecture has several advantages over other deeper networks including DenseNet; our architecture which we call Multipath-DenseNet is deeper as well as wider, reduces training time, and uses a smaller number of parameters. We evaluate our proposed architecture on the following four object recognition datasets: CIFAR-10, CIFAR-100, SVHN, and ImageNet. The evaluation results show that Multipath-DenseNet achieves significant improvement in performance over DenseNet on the benchmark datasets.

AB - Deep networks with skip-connections such as ResNets have achieved great results in recent years. DenseNet exploits the ResNet skip-connections by connecting each layer in convolution neural network to all preceding layers and achieves state-of-the-art accuracy. It is well-known that deeper networks are more efficient and easier to train than shallow or wider networks. Despite the high performance of very deep networks, they are limited in terms of vanishing gradient, diminishing forward flow, and slower training time. In this paper, we propose to combine the benefits of the depth and width of networks. We train supervised independent shallow networks on the same input in a block fashion. We use a state-of-the-art DenseNet block to increase the number of paths for gradient flow. Our proposed architecture has several advantages over other deeper networks including DenseNet; our architecture which we call Multipath-DenseNet is deeper as well as wider, reduces training time, and uses a smaller number of parameters. We evaluate our proposed architecture on the following four object recognition datasets: CIFAR-10, CIFAR-100, SVHN, and ImageNet. The evaluation results show that Multipath-DenseNet achieves significant improvement in performance over DenseNet on the benchmark datasets.

KW - Deep-learning

KW - Image classification

KW - Neural network

UR - http://www.scopus.com/inward/record.url?scp=85059736120&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059736120&partnerID=8YFLogxK

U2 - 10.1016/j.ins.2019.01.012

DO - 10.1016/j.ins.2019.01.012

M3 - Article

VL - 482

SP - 63

EP - 72

JO - Information Sciences

JF - Information Sciences

SN - 0020-0255

ER -