Path planning for a robot manipulator based on probabilistic roadmap and reinforcement learning

Jung Jun Park, Ji Hun Kim, Jae-Bok Song

Research output: Contribution to journalArticle

28 Citations (Scopus)

Abstract

The probabilistic roadmap (PRM) method, which is a popular path planning scheme, for a manipulator, can find a collision-free path by connecting the start and goal poses through a roadmap constructed by drawing random nodes in the free configuration space. PRM exhibits robust performance for static environments, but its performance is poor for dynamic environments. On the other hand, reinforcement learning, a behavior-based control technique, can deal with uncertainties in the environment. The reinforcement learning agent can establish a policy that maximizes the sum of rewards by selecting the optimal actions in any state through iterative interactions with the environment. In this paper, we propose efficient real-time path planning by combining PRM and reinforcement learning to deal with uncertain dynamic environments and similar environments. A series of experiments demonstrate that the proposed hybrid path planner can generate a collision-free path even for dynamic environments in which objects block the pre-planned global path. It is also shown that the hybrid path planner can adapt to the similar, previously learned environments without significant additional learning.

Original languageEnglish
Pages (from-to)674-680
Number of pages7
JournalInternational Journal of Control, Automation and Systems
Volume5
Issue number6
Publication statusPublished - 2007 Dec 1

Fingerprint

Reinforcement learning
Motion planning
Manipulators
Robots
Experiments

Keywords

  • Path planning
  • Probabilistic roadmap
  • Reinforcement learning
  • Robot manipulator

ASJC Scopus subject areas

  • Control and Systems Engineering

Cite this

Path planning for a robot manipulator based on probabilistic roadmap and reinforcement learning. / Park, Jung Jun; Kim, Ji Hun; Song, Jae-Bok.

In: International Journal of Control, Automation and Systems, Vol. 5, No. 6, 01.12.2007, p. 674-680.

Research output: Contribution to journalArticle

@article{ac6382384d404ee186063c680b07b28c,
title = "Path planning for a robot manipulator based on probabilistic roadmap and reinforcement learning",
abstract = "The probabilistic roadmap (PRM) method, which is a popular path planning scheme, for a manipulator, can find a collision-free path by connecting the start and goal poses through a roadmap constructed by drawing random nodes in the free configuration space. PRM exhibits robust performance for static environments, but its performance is poor for dynamic environments. On the other hand, reinforcement learning, a behavior-based control technique, can deal with uncertainties in the environment. The reinforcement learning agent can establish a policy that maximizes the sum of rewards by selecting the optimal actions in any state through iterative interactions with the environment. In this paper, we propose efficient real-time path planning by combining PRM and reinforcement learning to deal with uncertain dynamic environments and similar environments. A series of experiments demonstrate that the proposed hybrid path planner can generate a collision-free path even for dynamic environments in which objects block the pre-planned global path. It is also shown that the hybrid path planner can adapt to the similar, previously learned environments without significant additional learning.",
keywords = "Path planning, Probabilistic roadmap, Reinforcement learning, Robot manipulator",
author = "Park, {Jung Jun} and Kim, {Ji Hun} and Jae-Bok Song",
year = "2007",
month = "12",
day = "1",
language = "English",
volume = "5",
pages = "674--680",
journal = "International Journal of Control, Automation and Systems",
issn = "1598-6446",
publisher = "Institute of Control, Robotics and Systems",
number = "6",

}

TY - JOUR

T1 - Path planning for a robot manipulator based on probabilistic roadmap and reinforcement learning

AU - Park, Jung Jun

AU - Kim, Ji Hun

AU - Song, Jae-Bok

PY - 2007/12/1

Y1 - 2007/12/1

N2 - The probabilistic roadmap (PRM) method, which is a popular path planning scheme, for a manipulator, can find a collision-free path by connecting the start and goal poses through a roadmap constructed by drawing random nodes in the free configuration space. PRM exhibits robust performance for static environments, but its performance is poor for dynamic environments. On the other hand, reinforcement learning, a behavior-based control technique, can deal with uncertainties in the environment. The reinforcement learning agent can establish a policy that maximizes the sum of rewards by selecting the optimal actions in any state through iterative interactions with the environment. In this paper, we propose efficient real-time path planning by combining PRM and reinforcement learning to deal with uncertain dynamic environments and similar environments. A series of experiments demonstrate that the proposed hybrid path planner can generate a collision-free path even for dynamic environments in which objects block the pre-planned global path. It is also shown that the hybrid path planner can adapt to the similar, previously learned environments without significant additional learning.

AB - The probabilistic roadmap (PRM) method, which is a popular path planning scheme, for a manipulator, can find a collision-free path by connecting the start and goal poses through a roadmap constructed by drawing random nodes in the free configuration space. PRM exhibits robust performance for static environments, but its performance is poor for dynamic environments. On the other hand, reinforcement learning, a behavior-based control technique, can deal with uncertainties in the environment. The reinforcement learning agent can establish a policy that maximizes the sum of rewards by selecting the optimal actions in any state through iterative interactions with the environment. In this paper, we propose efficient real-time path planning by combining PRM and reinforcement learning to deal with uncertain dynamic environments and similar environments. A series of experiments demonstrate that the proposed hybrid path planner can generate a collision-free path even for dynamic environments in which objects block the pre-planned global path. It is also shown that the hybrid path planner can adapt to the similar, previously learned environments without significant additional learning.

KW - Path planning

KW - Probabilistic roadmap

KW - Reinforcement learning

KW - Robot manipulator

UR - http://www.scopus.com/inward/record.url?scp=36949014043&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=36949014043&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:36949014043

VL - 5

SP - 674

EP - 680

JO - International Journal of Control, Automation and Systems

JF - International Journal of Control, Automation and Systems

SN - 1598-6446

IS - 6

ER -