Personalized prediction of smartphone-based psychotherapeutic micro-intervention success using machine learning

Gunther Meinlschmidt, Marion Tegethoff, Angelo Belardi, Esther Stalujanis, Minkyung Oh, Eun Kyung Jung, Hyun Chul Kim, Seung Schik Yoo, Jong Hwan Lee

Research output: Contribution to journalArticle

Abstract

Background: Tailoring healthcare to patients’ individual needs is a central goal of precision medicine. Combining smartphone-based interventions with machine learning approaches may help attaining this goal. The aim of our study was to explore the predictability of the success of smartphone-based psychotherapeutic micro-interventions in eliciting mood changes using machine learning. Methods: Participants conducted daily smartphone-based psychotherapeutic micro-interventions, guided by short video clips, for 13 consecutive days. Participants chose one of four intervention techniques used in psychotherapeutic approaches. Mood changes were assessed using the Multidimensional Mood State Questionnaire. Micro-intervention success was predicted using random forest (RF) tree-based mixed-effects logistic regression models. Data from 27 participants were used, totaling 324 micro-interventions, randomly split 100 times into training and test samples, using within-subject and between-subject sampling. Results: Mood improved from pre- to post-intervention in 137 sessions (initial success-rate: 42.3%). The RF approach resulted in predictions of micro-intervention success significantly better than the initial success-rate within and between subjects (positive predictive value: 0.732 (95%-CI: 0.607; 0.820) and 0.698 (95%-CI: 0.564; 0.805), respectively). Prediction quality was highest using the RF approach within subjects (rand accuracy: 0.75 (95%-CI: 0.641; 0.840), Matthew's correlation coefficient: 0.483 (95%-CI: 0.323; 0.723)). Limitations: The RF approach does not allow firm conclusions about the exact contribution of each factor to the algorithm's predictions. We included a limited number of predictors and did not compare whether predictability differed between psychotherapeutic techniques. Conclusions: Our findings may pave the way for translation and encourage scrutinizing personalized prediction in the psychotherapeutic context to improve treatment efficacy.

Original languageEnglish
Pages (from-to)430-437
Number of pages8
JournalJournal of Affective Disorders
Volume264
DOIs
Publication statusPublished - 2020 Mar 1

Fingerprint

Logistic Models
Precision Medicine
Surgical Instruments
Delivery of Health Care
Smartphone
Machine Learning
Surveys and Questionnaires
Forests

Keywords

  • Binary classification
  • Ecological momentary intervention
  • Internet- and mobile-based intervention
  • Mental disorder
  • Mhealth
  • Supervised learning

ASJC Scopus subject areas

  • Clinical Psychology
  • Psychiatry and Mental health

Cite this

Personalized prediction of smartphone-based psychotherapeutic micro-intervention success using machine learning. / Meinlschmidt, Gunther; Tegethoff, Marion; Belardi, Angelo; Stalujanis, Esther; Oh, Minkyung; Jung, Eun Kyung; Kim, Hyun Chul; Yoo, Seung Schik; Lee, Jong Hwan.

In: Journal of Affective Disorders, Vol. 264, 01.03.2020, p. 430-437.

Research output: Contribution to journalArticle

Meinlschmidt, Gunther ; Tegethoff, Marion ; Belardi, Angelo ; Stalujanis, Esther ; Oh, Minkyung ; Jung, Eun Kyung ; Kim, Hyun Chul ; Yoo, Seung Schik ; Lee, Jong Hwan. / Personalized prediction of smartphone-based psychotherapeutic micro-intervention success using machine learning. In: Journal of Affective Disorders. 2020 ; Vol. 264. pp. 430-437.
@article{a8df383a8bc743c0a82bdb733c2530c4,
title = "Personalized prediction of smartphone-based psychotherapeutic micro-intervention success using machine learning",
abstract = "Background: Tailoring healthcare to patients’ individual needs is a central goal of precision medicine. Combining smartphone-based interventions with machine learning approaches may help attaining this goal. The aim of our study was to explore the predictability of the success of smartphone-based psychotherapeutic micro-interventions in eliciting mood changes using machine learning. Methods: Participants conducted daily smartphone-based psychotherapeutic micro-interventions, guided by short video clips, for 13 consecutive days. Participants chose one of four intervention techniques used in psychotherapeutic approaches. Mood changes were assessed using the Multidimensional Mood State Questionnaire. Micro-intervention success was predicted using random forest (RF) tree-based mixed-effects logistic regression models. Data from 27 participants were used, totaling 324 micro-interventions, randomly split 100 times into training and test samples, using within-subject and between-subject sampling. Results: Mood improved from pre- to post-intervention in 137 sessions (initial success-rate: 42.3{\%}). The RF approach resulted in predictions of micro-intervention success significantly better than the initial success-rate within and between subjects (positive predictive value: 0.732 (95{\%}-CI: 0.607; 0.820) and 0.698 (95{\%}-CI: 0.564; 0.805), respectively). Prediction quality was highest using the RF approach within subjects (rand accuracy: 0.75 (95{\%}-CI: 0.641; 0.840), Matthew's correlation coefficient: 0.483 (95{\%}-CI: 0.323; 0.723)). Limitations: The RF approach does not allow firm conclusions about the exact contribution of each factor to the algorithm's predictions. We included a limited number of predictors and did not compare whether predictability differed between psychotherapeutic techniques. Conclusions: Our findings may pave the way for translation and encourage scrutinizing personalized prediction in the psychotherapeutic context to improve treatment efficacy.",
keywords = "Binary classification, Ecological momentary intervention, Internet- and mobile-based intervention, Mental disorder, Mhealth, Supervised learning",
author = "Gunther Meinlschmidt and Marion Tegethoff and Angelo Belardi and Esther Stalujanis and Minkyung Oh and Jung, {Eun Kyung} and Kim, {Hyun Chul} and Yoo, {Seung Schik} and Lee, {Jong Hwan}",
year = "2020",
month = "3",
day = "1",
doi = "10.1016/j.jad.2019.11.071",
language = "English",
volume = "264",
pages = "430--437",
journal = "Journal of Affective Disorders",
issn = "0165-0327",
publisher = "Elsevier",

}

TY - JOUR

T1 - Personalized prediction of smartphone-based psychotherapeutic micro-intervention success using machine learning

AU - Meinlschmidt, Gunther

AU - Tegethoff, Marion

AU - Belardi, Angelo

AU - Stalujanis, Esther

AU - Oh, Minkyung

AU - Jung, Eun Kyung

AU - Kim, Hyun Chul

AU - Yoo, Seung Schik

AU - Lee, Jong Hwan

PY - 2020/3/1

Y1 - 2020/3/1

N2 - Background: Tailoring healthcare to patients’ individual needs is a central goal of precision medicine. Combining smartphone-based interventions with machine learning approaches may help attaining this goal. The aim of our study was to explore the predictability of the success of smartphone-based psychotherapeutic micro-interventions in eliciting mood changes using machine learning. Methods: Participants conducted daily smartphone-based psychotherapeutic micro-interventions, guided by short video clips, for 13 consecutive days. Participants chose one of four intervention techniques used in psychotherapeutic approaches. Mood changes were assessed using the Multidimensional Mood State Questionnaire. Micro-intervention success was predicted using random forest (RF) tree-based mixed-effects logistic regression models. Data from 27 participants were used, totaling 324 micro-interventions, randomly split 100 times into training and test samples, using within-subject and between-subject sampling. Results: Mood improved from pre- to post-intervention in 137 sessions (initial success-rate: 42.3%). The RF approach resulted in predictions of micro-intervention success significantly better than the initial success-rate within and between subjects (positive predictive value: 0.732 (95%-CI: 0.607; 0.820) and 0.698 (95%-CI: 0.564; 0.805), respectively). Prediction quality was highest using the RF approach within subjects (rand accuracy: 0.75 (95%-CI: 0.641; 0.840), Matthew's correlation coefficient: 0.483 (95%-CI: 0.323; 0.723)). Limitations: The RF approach does not allow firm conclusions about the exact contribution of each factor to the algorithm's predictions. We included a limited number of predictors and did not compare whether predictability differed between psychotherapeutic techniques. Conclusions: Our findings may pave the way for translation and encourage scrutinizing personalized prediction in the psychotherapeutic context to improve treatment efficacy.

AB - Background: Tailoring healthcare to patients’ individual needs is a central goal of precision medicine. Combining smartphone-based interventions with machine learning approaches may help attaining this goal. The aim of our study was to explore the predictability of the success of smartphone-based psychotherapeutic micro-interventions in eliciting mood changes using machine learning. Methods: Participants conducted daily smartphone-based psychotherapeutic micro-interventions, guided by short video clips, for 13 consecutive days. Participants chose one of four intervention techniques used in psychotherapeutic approaches. Mood changes were assessed using the Multidimensional Mood State Questionnaire. Micro-intervention success was predicted using random forest (RF) tree-based mixed-effects logistic regression models. Data from 27 participants were used, totaling 324 micro-interventions, randomly split 100 times into training and test samples, using within-subject and between-subject sampling. Results: Mood improved from pre- to post-intervention in 137 sessions (initial success-rate: 42.3%). The RF approach resulted in predictions of micro-intervention success significantly better than the initial success-rate within and between subjects (positive predictive value: 0.732 (95%-CI: 0.607; 0.820) and 0.698 (95%-CI: 0.564; 0.805), respectively). Prediction quality was highest using the RF approach within subjects (rand accuracy: 0.75 (95%-CI: 0.641; 0.840), Matthew's correlation coefficient: 0.483 (95%-CI: 0.323; 0.723)). Limitations: The RF approach does not allow firm conclusions about the exact contribution of each factor to the algorithm's predictions. We included a limited number of predictors and did not compare whether predictability differed between psychotherapeutic techniques. Conclusions: Our findings may pave the way for translation and encourage scrutinizing personalized prediction in the psychotherapeutic context to improve treatment efficacy.

KW - Binary classification

KW - Ecological momentary intervention

KW - Internet- and mobile-based intervention

KW - Mental disorder

KW - Mhealth

KW - Supervised learning

UR - http://www.scopus.com/inward/record.url?scp=85075906633&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075906633&partnerID=8YFLogxK

U2 - 10.1016/j.jad.2019.11.071

DO - 10.1016/j.jad.2019.11.071

M3 - Article

AN - SCOPUS:85075906633

VL - 264

SP - 430

EP - 437

JO - Journal of Affective Disorders

JF - Journal of Affective Disorders

SN - 0165-0327

ER -