Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm

Baeksuk Chu, Jooyoung Park, Daehie Hong

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Tunnel ventilation systems provide drivers with a comfortable and safe driving environment by generating sufficient airflow and by diluting the concentration of noxious contaminants below an acceptable level. For that purpose, tunnel ventilation systems contain mechanical equipment such as jet-fans, blowers and dust collectors. These machines consume large amount of energy, therefore, it is necessary to have an efficient operating algorithm for tunnel ventilation in terms of energy savings and safe driving. In this paper, a new reinforcement learning (RL) method is applied as the control algorithm. In the process of formulating the reward of the tunnel ventilation system, which is a performance index to be maximized in the RL methodology, the following two objectives are of great interest: maintaining an adequate level of pollutants and minimizing power consumption. The RL control algorithm adopted in this research is based on an actor-critic architecture and natural gradient method. Due to its ability to achieve the truly steepest direction of gradients, the natural gradient method can be a promising route to improving the efficacy of the actor module. Also, the recursive least-squares (RLS) method is employed to the critic module in order to improve the efficiency by which data is used. Using actual data collected from an existing tunnel ventilation system, extensive simulation studies were performed. It was confirmed that the suggested algorithm achieved the desired control goals and, when compared to previously developed RL-based control algorithms, improved the performance considerably.

Original languageEnglish
Pages (from-to)829-838
Number of pages10
JournalInternational Journal of Precision Engineering and Manufacturing
Volume11
Issue number6
DOIs
Publication statusPublished - 2010 Dec 1

Fingerprint

Ventilation
Reinforcement learning
Tunnels
Controllers
Gradient methods
Dust collectors
Blowers
Fans
Energy conservation
Electric power utilization
Impurities

Keywords

  • Actor-critic architecture
  • Natural gradient
  • Recursive least-squares (RLS)
  • Reinforcement learning (RL)
  • Tunnel ventilation control

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Industrial and Manufacturing Engineering
  • Mechanical Engineering

Cite this

Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm. / Chu, Baeksuk; Park, Jooyoung; Hong, Daehie.

In: International Journal of Precision Engineering and Manufacturing, Vol. 11, No. 6, 01.12.2010, p. 829-838.

Research output: Contribution to journalArticle

@article{b4a6ae629d794a2ab7e6894fb686ed50,
title = "Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm",
abstract = "Tunnel ventilation systems provide drivers with a comfortable and safe driving environment by generating sufficient airflow and by diluting the concentration of noxious contaminants below an acceptable level. For that purpose, tunnel ventilation systems contain mechanical equipment such as jet-fans, blowers and dust collectors. These machines consume large amount of energy, therefore, it is necessary to have an efficient operating algorithm for tunnel ventilation in terms of energy savings and safe driving. In this paper, a new reinforcement learning (RL) method is applied as the control algorithm. In the process of formulating the reward of the tunnel ventilation system, which is a performance index to be maximized in the RL methodology, the following two objectives are of great interest: maintaining an adequate level of pollutants and minimizing power consumption. The RL control algorithm adopted in this research is based on an actor-critic architecture and natural gradient method. Due to its ability to achieve the truly steepest direction of gradients, the natural gradient method can be a promising route to improving the efficacy of the actor module. Also, the recursive least-squares (RLS) method is employed to the critic module in order to improve the efficiency by which data is used. Using actual data collected from an existing tunnel ventilation system, extensive simulation studies were performed. It was confirmed that the suggested algorithm achieved the desired control goals and, when compared to previously developed RL-based control algorithms, improved the performance considerably.",
keywords = "Actor-critic architecture, Natural gradient, Recursive least-squares (RLS), Reinforcement learning (RL), Tunnel ventilation control",
author = "Baeksuk Chu and Jooyoung Park and Daehie Hong",
year = "2010",
month = "12",
day = "1",
doi = "10.1007/s12541-010-0100-6",
language = "English",
volume = "11",
pages = "829--838",
journal = "International Journal of Precision Engineering and Manufacturing",
issn = "1229-8557",
publisher = "Korean Society of Precision Engineering",
number = "6",

}

TY - JOUR

T1 - Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm

AU - Chu, Baeksuk

AU - Park, Jooyoung

AU - Hong, Daehie

PY - 2010/12/1

Y1 - 2010/12/1

N2 - Tunnel ventilation systems provide drivers with a comfortable and safe driving environment by generating sufficient airflow and by diluting the concentration of noxious contaminants below an acceptable level. For that purpose, tunnel ventilation systems contain mechanical equipment such as jet-fans, blowers and dust collectors. These machines consume large amount of energy, therefore, it is necessary to have an efficient operating algorithm for tunnel ventilation in terms of energy savings and safe driving. In this paper, a new reinforcement learning (RL) method is applied as the control algorithm. In the process of formulating the reward of the tunnel ventilation system, which is a performance index to be maximized in the RL methodology, the following two objectives are of great interest: maintaining an adequate level of pollutants and minimizing power consumption. The RL control algorithm adopted in this research is based on an actor-critic architecture and natural gradient method. Due to its ability to achieve the truly steepest direction of gradients, the natural gradient method can be a promising route to improving the efficacy of the actor module. Also, the recursive least-squares (RLS) method is employed to the critic module in order to improve the efficiency by which data is used. Using actual data collected from an existing tunnel ventilation system, extensive simulation studies were performed. It was confirmed that the suggested algorithm achieved the desired control goals and, when compared to previously developed RL-based control algorithms, improved the performance considerably.

AB - Tunnel ventilation systems provide drivers with a comfortable and safe driving environment by generating sufficient airflow and by diluting the concentration of noxious contaminants below an acceptable level. For that purpose, tunnel ventilation systems contain mechanical equipment such as jet-fans, blowers and dust collectors. These machines consume large amount of energy, therefore, it is necessary to have an efficient operating algorithm for tunnel ventilation in terms of energy savings and safe driving. In this paper, a new reinforcement learning (RL) method is applied as the control algorithm. In the process of formulating the reward of the tunnel ventilation system, which is a performance index to be maximized in the RL methodology, the following two objectives are of great interest: maintaining an adequate level of pollutants and minimizing power consumption. The RL control algorithm adopted in this research is based on an actor-critic architecture and natural gradient method. Due to its ability to achieve the truly steepest direction of gradients, the natural gradient method can be a promising route to improving the efficacy of the actor module. Also, the recursive least-squares (RLS) method is employed to the critic module in order to improve the efficiency by which data is used. Using actual data collected from an existing tunnel ventilation system, extensive simulation studies were performed. It was confirmed that the suggested algorithm achieved the desired control goals and, when compared to previously developed RL-based control algorithms, improved the performance considerably.

KW - Actor-critic architecture

KW - Natural gradient

KW - Recursive least-squares (RLS)

KW - Reinforcement learning (RL)

KW - Tunnel ventilation control

UR - http://www.scopus.com/inward/record.url?scp=78649967421&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78649967421&partnerID=8YFLogxK

U2 - 10.1007/s12541-010-0100-6

DO - 10.1007/s12541-010-0100-6

M3 - Article

AN - SCOPUS:78649967421

VL - 11

SP - 829

EP - 838

JO - International Journal of Precision Engineering and Manufacturing

JF - International Journal of Precision Engineering and Manufacturing

SN - 1229-8557

IS - 6

ER -