Parallelizing parallel rollout algorithm for solving Markov decision processes

Seon Wook Kim, Hyeong Soo Chang

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Parallel rollout is a formal method of combining multiple heuristic policies available to a sequential decision maker in the framework of Markov Decision Processes (MDPs). The method improves the performances of all of the heuristic policies adapting to the different stochastic system trajectories. From an inherent multi-level parallelism in the method, in this paper we propose a parallelized version of parallel rollout algorithm, and evaluate its performance on a multi-class task scheduling problem by using OpenMP and MPI programming model. We analyze and compare the performance in two versions of parallelized codes, e.g., OpenMP and MPI on several execution environment. We show that the performance using OpenMP API is higher than MPI due to lower overhead in data synchronization across processors.

Original languageEnglish
Pages (from-to)122-136
Number of pages15
JournalLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2716
Publication statusPublished - 2003 Dec 1

Fingerprint

Markov Chains
Formal methods
Stochastic systems
Markov Decision Process
Application programming interfaces (API)
Parallel algorithms
Parallel Algorithms
OpenMP
Synchronization
Scheduling
Trajectories
Heuristics
Task Scheduling
Formal Methods
Multi-class
Stochastic Systems
Programming Model
Parallelism
Scheduling Problem
Trajectory

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Computer Science(all)
  • Theoretical Computer Science

Cite this

@article{5726466985a440f8afb3c02de029e5dc,
title = "Parallelizing parallel rollout algorithm for solving Markov decision processes",
abstract = "Parallel rollout is a formal method of combining multiple heuristic policies available to a sequential decision maker in the framework of Markov Decision Processes (MDPs). The method improves the performances of all of the heuristic policies adapting to the different stochastic system trajectories. From an inherent multi-level parallelism in the method, in this paper we propose a parallelized version of parallel rollout algorithm, and evaluate its performance on a multi-class task scheduling problem by using OpenMP and MPI programming model. We analyze and compare the performance in two versions of parallelized codes, e.g., OpenMP and MPI on several execution environment. We show that the performance using OpenMP API is higher than MPI due to lower overhead in data synchronization across processors.",
author = "Kim, {Seon Wook} and Chang, {Hyeong Soo}",
year = "2003",
month = "12",
day = "1",
language = "English",
volume = "2716",
pages = "122--136",
journal = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
issn = "0302-9743",
publisher = "Springer Verlag",

}

TY - JOUR

T1 - Parallelizing parallel rollout algorithm for solving Markov decision processes

AU - Kim, Seon Wook

AU - Chang, Hyeong Soo

PY - 2003/12/1

Y1 - 2003/12/1

N2 - Parallel rollout is a formal method of combining multiple heuristic policies available to a sequential decision maker in the framework of Markov Decision Processes (MDPs). The method improves the performances of all of the heuristic policies adapting to the different stochastic system trajectories. From an inherent multi-level parallelism in the method, in this paper we propose a parallelized version of parallel rollout algorithm, and evaluate its performance on a multi-class task scheduling problem by using OpenMP and MPI programming model. We analyze and compare the performance in two versions of parallelized codes, e.g., OpenMP and MPI on several execution environment. We show that the performance using OpenMP API is higher than MPI due to lower overhead in data synchronization across processors.

AB - Parallel rollout is a formal method of combining multiple heuristic policies available to a sequential decision maker in the framework of Markov Decision Processes (MDPs). The method improves the performances of all of the heuristic policies adapting to the different stochastic system trajectories. From an inherent multi-level parallelism in the method, in this paper we propose a parallelized version of parallel rollout algorithm, and evaluate its performance on a multi-class task scheduling problem by using OpenMP and MPI programming model. We analyze and compare the performance in two versions of parallelized codes, e.g., OpenMP and MPI on several execution environment. We show that the performance using OpenMP API is higher than MPI due to lower overhead in data synchronization across processors.

UR - http://www.scopus.com/inward/record.url?scp=35248881097&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=35248881097&partnerID=8YFLogxK

M3 - Article

VL - 2716

SP - 122

EP - 136

JO - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

JF - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SN - 0302-9743

ER -