Shallow Fully Connected Neural Network Training by Forcing Linearization into Valid Region and Balancing Training Rates

Jea Pil Heo, Chang Gyu Im, Kyung Hwan Ryu, Su Whan Sung, Changkyoo Yoo, Dae Ryook Yang

Research output: Contribution to journalArticlepeer-review

Abstract

A new supervisory training rule for a shallow fully connected neural network (SFCNN) is proposed in this present study. The proposed training rule is developed based on local linearization and analytical optimal solutions for linearized SFCNN. The cause of nonlinearity in neural network training is analyzed, and it is removed by local linearization. The optimal solution for the linearized SFCNN, which minimizes the cost function for the training, is analytically derived. Additionally, the training efficiency and model accuracy of the trained SFCNN are improved by keeping estimates within a valid range of the linearization. The superiority of the proposed approach is demonstrated by applying the proposed training rule to the modeling of a typical nonlinear pH process, Boston housing prices dataset, and automobile mileage per gallon dataset. The proposed training rule shows the smallest modeling error and the smallest iteration number required for convergence compared with several previous approaches from the literature for the case study.

Original languageEnglish
Article number1157
JournalProcesses
Volume10
Issue number6
DOIs
Publication statusPublished - 2022 Jun

Keywords

  • local linearization
  • neural network
  • optimal solution
  • pH system modeling
  • training rule

ASJC Scopus subject areas

  • Bioengineering
  • Chemical Engineering (miscellaneous)
  • Process Chemistry and Technology

Fingerprint

Dive into the research topics of 'Shallow Fully Connected Neural Network Training by Forcing Linearization into Valid Region and Balancing Training Rates'. Together they form a unique fingerprint.

Cite this