Enhancing monte carlo tree search for playing hearthstone

Jean Seong Bjorn Choe, Jong Kook Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Hearthstone is a popular online collectible card game (CCG). Hearthstone imposes interesting challenges in developing a search algorithm for the game AI. As a CCG, it has a considerable amount of hidden information from each player's private hand and deck. Moreover, the action space is full of stochastic actions compared to other similar games. That is, instead of a single move, each player is allowed to build a move sequence via various combinations of atomic actions. Therefore, when applying any heuristic search algorithm, the branching factor of the search space is extremely large. In this paper, we explore the use of Monte Carlo Tree Search (MCTS) with approaches to reduce the complexity of the search space and decide on the best strategy. First, we utilise state abstraction to present the search space as a Directed Acyclic Graph (DAG) and introduce a variant of Upper Confidence Bound for Trees (UCT) algorithm for the DAG. Next, we apply the sparse sampling algorithm to handle imperfect information and randomness and reduce the stochastic branching factor. This paper presents empirical evaluations of the proposed framework for Hearthstone and the experimental results suggest that our approach is well suited for developing a better AI agent.

Original languageEnglish
Title of host publicationIEEE Conference on Games 2019, CoG 2019
PublisherIEEE Computer Society
ISBN (Electronic)9781728118840
DOIs
Publication statusPublished - 2019 Aug
Event2019 IEEE Conference on Games, CoG 2019 - London, United Kingdom
Duration: 2019 Aug 202019 Aug 23

Publication series

NameIEEE Conference on Computatonal Intelligence and Games, CIG
Volume2019-August
ISSN (Print)2325-4270
ISSN (Electronic)2325-4289

Conference

Conference2019 IEEE Conference on Games, CoG 2019
CountryUnited Kingdom
CityLondon
Period19/8/2019/8/23

Keywords

  • Artificial intelligence for games
  • Hearthstone
  • Monte-Carlo tree search

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Fingerprint Dive into the research topics of 'Enhancing monte carlo tree search for playing hearthstone'. Together they form a unique fingerprint.

  • Cite this

    Choe, J. S. B., & Kim, J. K. (2019). Enhancing monte carlo tree search for playing hearthstone. In IEEE Conference on Games 2019, CoG 2019 [8848034] (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2019-August). IEEE Computer Society. https://doi.org/10.1109/CIG.2019.8848034