Human activity prediction based on Sub-volume Relationship Descriptor

Dong Gyu Lee, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

In this paper, we address the problem of recognizing unfinished human activity from partially observed videos. Specifically, we propose a novel human activity descriptor, which can represent pairwise relationships among human activities in a compact manner using pre-trained Convolutional Neural Networks (CNNs) by capturing the discriminative sub-volume. The potentially important relationship among all pairwise sub-volumes, called key-volumes, is automatically captured using global and local motion activation and the ratio of the participant. The captured key-volumes without prior knowledge hold discriminative information related to the unfinished activity. The key-volume information is considered in the descriptor construction procedure. Training a CNN model for a particular purpose requires a lot of resources, such as large amount of labeled data and computing power, despite its representational power. Thus, we develop a method to utilize pre-trained CNN without any additional model training procedure. The low-level features can be extracted through existing CNN toolkits. For a real application, the proposed method may be more cost-effective while implementing a smart surveillance system to understand human activity. In our experiments, we compare the performances of the proposed method with other state-of-the-art human activity prediction methods for two public datasets; the results of the experiments show that the proposed method outperforms these competing methods.

Original languageEnglish
Title of host publication2016 23rd International Conference on Pattern Recognition, ICPR 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2060-2065
Number of pages6
ISBN (Electronic)9781509048472
DOIs
Publication statusPublished - 2017 Apr 13
Event23rd International Conference on Pattern Recognition, ICPR 2016 - Cancun, Mexico
Duration: 2016 Dec 42016 Dec 8

Other

Other23rd International Conference on Pattern Recognition, ICPR 2016
CountryMexico
CityCancun
Period16/12/416/12/8

Fingerprint

Neural networks
Experiments
Chemical activation
Costs

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Cite this

Lee, D. G., & Lee, S. W. (2017). Human activity prediction based on Sub-volume Relationship Descriptor. In 2016 23rd International Conference on Pattern Recognition, ICPR 2016 (pp. 2060-2065). [7899939] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICPR.2016.7899939

Human activity prediction based on Sub-volume Relationship Descriptor. / Lee, Dong Gyu; Lee, Seong Whan.

2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2060-2065 7899939.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Lee, DG & Lee, SW 2017, Human activity prediction based on Sub-volume Relationship Descriptor. in 2016 23rd International Conference on Pattern Recognition, ICPR 2016., 7899939, Institute of Electrical and Electronics Engineers Inc., pp. 2060-2065, 23rd International Conference on Pattern Recognition, ICPR 2016, Cancun, Mexico, 16/12/4. https://doi.org/10.1109/ICPR.2016.7899939
Lee DG, Lee SW. Human activity prediction based on Sub-volume Relationship Descriptor. In 2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 2060-2065. 7899939 https://doi.org/10.1109/ICPR.2016.7899939
Lee, Dong Gyu ; Lee, Seong Whan. / Human activity prediction based on Sub-volume Relationship Descriptor. 2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 2060-2065
@inproceedings{835920f436d644fe8fb16017e9ef2cb3,
title = "Human activity prediction based on Sub-volume Relationship Descriptor",
abstract = "In this paper, we address the problem of recognizing unfinished human activity from partially observed videos. Specifically, we propose a novel human activity descriptor, which can represent pairwise relationships among human activities in a compact manner using pre-trained Convolutional Neural Networks (CNNs) by capturing the discriminative sub-volume. The potentially important relationship among all pairwise sub-volumes, called key-volumes, is automatically captured using global and local motion activation and the ratio of the participant. The captured key-volumes without prior knowledge hold discriminative information related to the unfinished activity. The key-volume information is considered in the descriptor construction procedure. Training a CNN model for a particular purpose requires a lot of resources, such as large amount of labeled data and computing power, despite its representational power. Thus, we develop a method to utilize pre-trained CNN without any additional model training procedure. The low-level features can be extracted through existing CNN toolkits. For a real application, the proposed method may be more cost-effective while implementing a smart surveillance system to understand human activity. In our experiments, we compare the performances of the proposed method with other state-of-the-art human activity prediction methods for two public datasets; the results of the experiments show that the proposed method outperforms these competing methods.",
author = "Lee, {Dong Gyu} and Lee, {Seong Whan}",
year = "2017",
month = "4",
day = "13",
doi = "10.1109/ICPR.2016.7899939",
language = "English",
pages = "2060--2065",
booktitle = "2016 23rd International Conference on Pattern Recognition, ICPR 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Human activity prediction based on Sub-volume Relationship Descriptor

AU - Lee, Dong Gyu

AU - Lee, Seong Whan

PY - 2017/4/13

Y1 - 2017/4/13

N2 - In this paper, we address the problem of recognizing unfinished human activity from partially observed videos. Specifically, we propose a novel human activity descriptor, which can represent pairwise relationships among human activities in a compact manner using pre-trained Convolutional Neural Networks (CNNs) by capturing the discriminative sub-volume. The potentially important relationship among all pairwise sub-volumes, called key-volumes, is automatically captured using global and local motion activation and the ratio of the participant. The captured key-volumes without prior knowledge hold discriminative information related to the unfinished activity. The key-volume information is considered in the descriptor construction procedure. Training a CNN model for a particular purpose requires a lot of resources, such as large amount of labeled data and computing power, despite its representational power. Thus, we develop a method to utilize pre-trained CNN without any additional model training procedure. The low-level features can be extracted through existing CNN toolkits. For a real application, the proposed method may be more cost-effective while implementing a smart surveillance system to understand human activity. In our experiments, we compare the performances of the proposed method with other state-of-the-art human activity prediction methods for two public datasets; the results of the experiments show that the proposed method outperforms these competing methods.

AB - In this paper, we address the problem of recognizing unfinished human activity from partially observed videos. Specifically, we propose a novel human activity descriptor, which can represent pairwise relationships among human activities in a compact manner using pre-trained Convolutional Neural Networks (CNNs) by capturing the discriminative sub-volume. The potentially important relationship among all pairwise sub-volumes, called key-volumes, is automatically captured using global and local motion activation and the ratio of the participant. The captured key-volumes without prior knowledge hold discriminative information related to the unfinished activity. The key-volume information is considered in the descriptor construction procedure. Training a CNN model for a particular purpose requires a lot of resources, such as large amount of labeled data and computing power, despite its representational power. Thus, we develop a method to utilize pre-trained CNN without any additional model training procedure. The low-level features can be extracted through existing CNN toolkits. For a real application, the proposed method may be more cost-effective while implementing a smart surveillance system to understand human activity. In our experiments, we compare the performances of the proposed method with other state-of-the-art human activity prediction methods for two public datasets; the results of the experiments show that the proposed method outperforms these competing methods.

UR - http://www.scopus.com/inward/record.url?scp=85019141877&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019141877&partnerID=8YFLogxK

U2 - 10.1109/ICPR.2016.7899939

DO - 10.1109/ICPR.2016.7899939

M3 - Conference contribution

AN - SCOPUS:85019141877

SP - 2060

EP - 2065

BT - 2016 23rd International Conference on Pattern Recognition, ICPR 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -