Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning

Juyoung Lee; Andreas Pastor; Jae In Hwang; Gerard Jounghyun Kim

doi:10.1145/3359996.3364709

Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning

Juyoung Lee, Andreas Pastor, Jae In Hwang, Gerard Jounghyun Kim

Department of Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we propose to use the deep learning technique to estimate and predict the torso direction from the head movements alone. The prediction allows to implement the walk-in-place navigation interface without additional sensing of the torso direction, and thereby improves the convenience and usability. We created a small dataset and tested our idea by training an LSTM model and obtained a 3-class prediction rate of about 90%, a figure higher than using other conventional machine learning techniques. While preliminary, the results show the possible inter-dependence between the viewing and torso directions, and with richer dataset and more parameters, a more accurate level of prediction seems possible.

Original language	English
Title of host publication	Proceedings - VRST 2019
Subtitle of host publication	25th ACM Symposium on Virtual Reality Software and Technology
Editors	Stephen N. Spencer
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450370011
DOIs	https://doi.org/10.1145/3359996.3364709
Publication status	Published - 2019 Nov 12
Event	25th ACM Symposium on Virtual Reality Software and Technology, VRST 2019 - Sydney, Australia Duration: 2019 Nov 12 → 2019 Nov 15

Publication series

Name	Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST

Conference

Conference	25th ACM Symposium on Virtual Reality Software and Technology, VRST 2019
Country	Australia
City	Sydney
Period	19/11/12 → 19/11/15

Keywords

Deep learning
Locomotion
Virtual reality
Walking in place

ASJC Scopus subject areas

Software

Access to Document

10.1145/3359996.3364709

Fingerprint Dive into the research topics of 'Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning'. Together they form a unique fingerprint.

Cite this

Lee, J., Pastor, A., Hwang, J. I., & Kim, G. J. (2019). Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning. In S. N. Spencer (Ed.), Proceedings - VRST 2019: 25th ACM Symposium on Virtual Reality Software and Technology [3364709] (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST). Association for Computing Machinery. https://doi.org/10.1145/3359996.3364709

Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning. / Lee, Juyoung; Pastor, Andreas; Hwang, Jae In; Kim, Gerard Jounghyun.

Proceedings - VRST 2019: 25th ACM Symposium on Virtual Reality Software and Technology. ed. / Stephen N. Spencer. Association for Computing Machinery, 2019. 3364709 (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lee, J, Pastor, A, Hwang, JI & Kim, GJ 2019, Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning. in SN Spencer (ed.), Proceedings - VRST 2019: 25th ACM Symposium on Virtual Reality Software and Technology., 3364709, Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST, Association for Computing Machinery, 25th ACM Symposium on Virtual Reality Software and Technology, VRST 2019, Sydney, Australia, 19/11/12. https://doi.org/10.1145/3359996.3364709

Lee J, Pastor A, Hwang JI, Kim GJ. Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning. In Spencer SN, editor, Proceedings - VRST 2019: 25th ACM Symposium on Virtual Reality Software and Technology. Association for Computing Machinery. 2019. 3364709. (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST). https://doi.org/10.1145/3359996.3364709

Lee, Juyoung ; Pastor, Andreas ; Hwang, Jae In ; Kim, Gerard Jounghyun. / Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning. Proceedings - VRST 2019: 25th ACM Symposium on Virtual Reality Software and Technology. editor / Stephen N. Spencer. Association for Computing Machinery, 2019. (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST).

@inproceedings{96bebadf0cd54de1a9c4c91b6c3ef2d9,

title = "Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning",

abstract = "In this paper, we propose to use the deep learning technique to estimate and predict the torso direction from the head movements alone. The prediction allows to implement the walk-in-place navigation interface without additional sensing of the torso direction, and thereby improves the convenience and usability. We created a small dataset and tested our idea by training an LSTM model and obtained a 3-class prediction rate of about 90%, a figure higher than using other conventional machine learning techniques. While preliminary, the results show the possible inter-dependence between the viewing and torso directions, and with richer dataset and more parameters, a more accurate level of prediction seems possible.",

keywords = "Deep learning, Locomotion, Virtual reality, Walking in place",

author = "Juyoung Lee and Andreas Pastor and Hwang, {Jae In} and Kim, {Gerard Jounghyun}",

year = "2019",

month = nov,

day = "12",

doi = "10.1145/3359996.3364709",

language = "English",

series = "Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST",

publisher = "Association for Computing Machinery",

editor = "Spencer, {Stephen N.}",

booktitle = "Proceedings - VRST 2019",

note = "25th ACM Symposium on Virtual Reality Software and Technology, VRST 2019 ; Conference date: 12-11-2019 Through 15-11-2019",

}

TY - GEN

T1 - Predicting the torso direction from HMD movements for walk-in-place navigation through deep learning

AU - Lee, Juyoung

AU - Pastor, Andreas

AU - Hwang, Jae In

AU - Kim, Gerard Jounghyun

PY - 2019/11/12

Y1 - 2019/11/12

N2 - In this paper, we propose to use the deep learning technique to estimate and predict the torso direction from the head movements alone. The prediction allows to implement the walk-in-place navigation interface without additional sensing of the torso direction, and thereby improves the convenience and usability. We created a small dataset and tested our idea by training an LSTM model and obtained a 3-class prediction rate of about 90%, a figure higher than using other conventional machine learning techniques. While preliminary, the results show the possible inter-dependence between the viewing and torso directions, and with richer dataset and more parameters, a more accurate level of prediction seems possible.

AB - In this paper, we propose to use the deep learning technique to estimate and predict the torso direction from the head movements alone. The prediction allows to implement the walk-in-place navigation interface without additional sensing of the torso direction, and thereby improves the convenience and usability. We created a small dataset and tested our idea by training an LSTM model and obtained a 3-class prediction rate of about 90%, a figure higher than using other conventional machine learning techniques. While preliminary, the results show the possible inter-dependence between the viewing and torso directions, and with richer dataset and more parameters, a more accurate level of prediction seems possible.

KW - Deep learning

KW - Locomotion

KW - Virtual reality

KW - Walking in place

UR - http://www.scopus.com/inward/record.url?scp=85076160498&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076160498&partnerID=8YFLogxK

U2 - 10.1145/3359996.3364709

DO - 10.1145/3359996.3364709

M3 - Conference contribution

AN - SCOPUS:85076160498

T3 - Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST

BT - Proceedings - VRST 2019

A2 - Spencer, Stephen N.

PB - Association for Computing Machinery

T2 - 25th ACM Symposium on Virtual Reality Software and Technology, VRST 2019

Y2 - 12 November 2019 through 15 November 2019

ER -