3-D virtual studio for natural inter-"acting"

Namgyu Kim, Woontack Woo, Gerard J. Kim, Chan Mo Park

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

Virtual studios have long been used in commercial broadcasting. However, most virtual studios are based on "blue screen" technology, and its two-dimensional (2-D) nature restricts the user from making natural three-dimensional (3-D) interactions. Actors have to follow prewritten scripts and pretend as if directly interacting with the synthetic objects. This often creates an unnatural and seemingly uncoordinated output. In this paper, we introduce an improved virtual-studio framework to enable actors/users to interact in 3-D more naturally with the synthetic environment and objects. The proposed system uses a stereo camera to first construct a 3-D environment (for the actor to act in), a multiview camera to extract the image and 3-D information about the actor, and a real-time registration and rendering software for generating the final output. Synthetic 3-D objects can be easily inserted and rendered, in real time, together with the 3-D environment and video actor for natural 3-D interaction. The enabling of natural 3-D interaction would make more cinematic techniques possible including live and spontaneous acting. The proposed system is not limited to broadcast production, but can also be used for creating virtual/augmented-reality environments for training and entertainment.

Original languageEnglish
Pages (from-to)758-773
Number of pages16
JournalIEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans
Volume36
Issue number4
DOIs
Publication statusPublished - 2006 Jul
Externally publishedYes

    Fingerprint

Keywords

  • Augmented reality
  • Calibration
  • Image segmentation
  • Multiview and stereo cameras
  • Registration
  • Three-dimensional (3-D) interaction
  • Three-dimensional (3-D) model reconstruction
  • Virtual environment (VE)
  • Virtual studio
  • Z-keying

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this