Enhancing monte carlo tree search for playing hearthstone

Jean Seong Bjorn Choe, Jong Kook Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Hearthstone is a popular online collectible card game (CCG). Hearthstone imposes interesting challenges in developing a search algorithm for the game AI. As a CCG, it has a considerable amount of hidden information from each player's private hand and deck. Moreover, the action space is full of stochastic actions compared to other similar games. That is, instead of a single move, each player is allowed to build a move sequence via various combinations of atomic actions. Therefore, when applying any heuristic search algorithm, the branching factor of the search space is extremely large. In this paper, we explore the use of Monte Carlo Tree Search (MCTS) with approaches to reduce the complexity of the search space and decide on the best strategy. First, we utilise state abstraction to present the search space as a Directed Acyclic Graph (DAG) and introduce a variant of Upper Confidence Bound for Trees (UCT) algorithm for the DAG. Next, we apply the sparse sampling algorithm to handle imperfect information and randomness and reduce the stochastic branching factor. This paper presents empirical evaluations of the proposed framework for Hearthstone and the experimental results suggest that our approach is well suited for developing a better AI agent.

Original languageEnglish
Title of host publicationIEEE Conference on Games 2019, CoG 2019
PublisherIEEE Computer Society
ISBN (Electronic)9781728118840
DOIs
Publication statusPublished - 2019 Aug
Event2019 IEEE Conference on Games, CoG 2019 - London, United Kingdom
Duration: 2019 Aug 202019 Aug 23

Publication series

NameIEEE Conference on Computatonal Intelligence and Games, CIG
Volume2019-August
ISSN (Print)2325-4270
ISSN (Electronic)2325-4289

Conference

Conference2019 IEEE Conference on Games, CoG 2019
CountryUnited Kingdom
CityLondon
Period19/8/2019/8/23

Fingerprint

Sampling

Keywords

  • Artificial intelligence for games
  • Hearthstone
  • Monte-Carlo tree search

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Cite this

Choe, J. S. B., & Kim, J. K. (2019). Enhancing monte carlo tree search for playing hearthstone. In IEEE Conference on Games 2019, CoG 2019 [8848034] (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2019-August). IEEE Computer Society. https://doi.org/10.1109/CIG.2019.8848034

Enhancing monte carlo tree search for playing hearthstone. / Choe, Jean Seong Bjorn; Kim, Jong Kook.

IEEE Conference on Games 2019, CoG 2019. IEEE Computer Society, 2019. 8848034 (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2019-August).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Choe, JSB & Kim, JK 2019, Enhancing monte carlo tree search for playing hearthstone. in IEEE Conference on Games 2019, CoG 2019., 8848034, IEEE Conference on Computatonal Intelligence and Games, CIG, vol. 2019-August, IEEE Computer Society, 2019 IEEE Conference on Games, CoG 2019, London, United Kingdom, 19/8/20. https://doi.org/10.1109/CIG.2019.8848034
Choe JSB, Kim JK. Enhancing monte carlo tree search for playing hearthstone. In IEEE Conference on Games 2019, CoG 2019. IEEE Computer Society. 2019. 8848034. (IEEE Conference on Computatonal Intelligence and Games, CIG). https://doi.org/10.1109/CIG.2019.8848034
Choe, Jean Seong Bjorn ; Kim, Jong Kook. / Enhancing monte carlo tree search for playing hearthstone. IEEE Conference on Games 2019, CoG 2019. IEEE Computer Society, 2019. (IEEE Conference on Computatonal Intelligence and Games, CIG).
@inproceedings{ce2b3e10d6a044acb3d2f9a2df4020e0,
title = "Enhancing monte carlo tree search for playing hearthstone",
abstract = "Hearthstone is a popular online collectible card game (CCG). Hearthstone imposes interesting challenges in developing a search algorithm for the game AI. As a CCG, it has a considerable amount of hidden information from each player's private hand and deck. Moreover, the action space is full of stochastic actions compared to other similar games. That is, instead of a single move, each player is allowed to build a move sequence via various combinations of atomic actions. Therefore, when applying any heuristic search algorithm, the branching factor of the search space is extremely large. In this paper, we explore the use of Monte Carlo Tree Search (MCTS) with approaches to reduce the complexity of the search space and decide on the best strategy. First, we utilise state abstraction to present the search space as a Directed Acyclic Graph (DAG) and introduce a variant of Upper Confidence Bound for Trees (UCT) algorithm for the DAG. Next, we apply the sparse sampling algorithm to handle imperfect information and randomness and reduce the stochastic branching factor. This paper presents empirical evaluations of the proposed framework for Hearthstone and the experimental results suggest that our approach is well suited for developing a better AI agent.",
keywords = "Artificial intelligence for games, Hearthstone, Monte-Carlo tree search",
author = "Choe, {Jean Seong Bjorn} and Kim, {Jong Kook}",
year = "2019",
month = "8",
doi = "10.1109/CIG.2019.8848034",
language = "English",
series = "IEEE Conference on Computatonal Intelligence and Games, CIG",
publisher = "IEEE Computer Society",
booktitle = "IEEE Conference on Games 2019, CoG 2019",

}

TY - GEN

T1 - Enhancing monte carlo tree search for playing hearthstone

AU - Choe, Jean Seong Bjorn

AU - Kim, Jong Kook

PY - 2019/8

Y1 - 2019/8

N2 - Hearthstone is a popular online collectible card game (CCG). Hearthstone imposes interesting challenges in developing a search algorithm for the game AI. As a CCG, it has a considerable amount of hidden information from each player's private hand and deck. Moreover, the action space is full of stochastic actions compared to other similar games. That is, instead of a single move, each player is allowed to build a move sequence via various combinations of atomic actions. Therefore, when applying any heuristic search algorithm, the branching factor of the search space is extremely large. In this paper, we explore the use of Monte Carlo Tree Search (MCTS) with approaches to reduce the complexity of the search space and decide on the best strategy. First, we utilise state abstraction to present the search space as a Directed Acyclic Graph (DAG) and introduce a variant of Upper Confidence Bound for Trees (UCT) algorithm for the DAG. Next, we apply the sparse sampling algorithm to handle imperfect information and randomness and reduce the stochastic branching factor. This paper presents empirical evaluations of the proposed framework for Hearthstone and the experimental results suggest that our approach is well suited for developing a better AI agent.

AB - Hearthstone is a popular online collectible card game (CCG). Hearthstone imposes interesting challenges in developing a search algorithm for the game AI. As a CCG, it has a considerable amount of hidden information from each player's private hand and deck. Moreover, the action space is full of stochastic actions compared to other similar games. That is, instead of a single move, each player is allowed to build a move sequence via various combinations of atomic actions. Therefore, when applying any heuristic search algorithm, the branching factor of the search space is extremely large. In this paper, we explore the use of Monte Carlo Tree Search (MCTS) with approaches to reduce the complexity of the search space and decide on the best strategy. First, we utilise state abstraction to present the search space as a Directed Acyclic Graph (DAG) and introduce a variant of Upper Confidence Bound for Trees (UCT) algorithm for the DAG. Next, we apply the sparse sampling algorithm to handle imperfect information and randomness and reduce the stochastic branching factor. This paper presents empirical evaluations of the proposed framework for Hearthstone and the experimental results suggest that our approach is well suited for developing a better AI agent.

KW - Artificial intelligence for games

KW - Hearthstone

KW - Monte-Carlo tree search

UR - http://www.scopus.com/inward/record.url?scp=85073115552&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073115552&partnerID=8YFLogxK

U2 - 10.1109/CIG.2019.8848034

DO - 10.1109/CIG.2019.8848034

M3 - Conference contribution

AN - SCOPUS:85073115552

T3 - IEEE Conference on Computatonal Intelligence and Games, CIG

BT - IEEE Conference on Games 2019, CoG 2019

PB - IEEE Computer Society

ER -