RIDE: real-time massive image processing platform on distributed environment

Yoon Ki Kim, Yongsung Kim, Chang-Sung Jeong

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

As the demand for real-time data processing increases, a high-speed processing platform for large-scale stream data becomes necessary. For fast processing large-scale stream data, it is essential to use multiple distributed nodes. So far, there have been few studies on real-time massive image processing through efficient management and allocation of heterogeneous resources for various user-specified nodes on distributed environments. In this paper, we shall present a new platform called RIDE (Real-time massive Image processing platform on Distributed Environment) which efficiently allocates resources and executes load balancing according to the amount of stream data on distributed environments. It minimizes communication overhead by using a parallel processing strategy which handles the stream data considering both coarse-grained and fine-grained parallelism simultaneously. Coarse-grained parallelism is achieved by the automatic allocation of input streams onto partitions of broker buffer each processed by its corresponding worker node, and maximized by adaptive resource management which adjusts the number of worker nodes in a group according to the frame rate in real time. Fine-grained parallelism is achieved by parallel processing of task on each worker node and maximized by allocating heterogeneous resources such as GPU and embedded machines appropriately. Moreover, it provides a scheme of application topology which has a great advantage for higher performance by configuring the worker nodes of each stage using adaptive heterogeneous resource management. Finally, it supports dynamic fault tolerance for real-time image processing through the coordination between components in our system.

Original languageEnglish
Article number39
JournalEurasip Journal on Image and Video Processing
Volume2018
Issue number1
DOIs
Publication statusPublished - 2018 Dec 1

Fingerprint

Image processing
Processing
Fault tolerance
Resource allocation
Topology
Communication

Keywords

  • Distributed and parallel processing
  • Heterogeneous computing
  • Image processing
  • Real-time

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Electrical and Electronic Engineering

Cite this

RIDE : real-time massive image processing platform on distributed environment. / Kim, Yoon Ki; Kim, Yongsung; Jeong, Chang-Sung.

In: Eurasip Journal on Image and Video Processing, Vol. 2018, No. 1, 39, 01.12.2018.

Research output: Contribution to journalArticle

@article{c8b24687c8fb499cb9cbaeed745cc1c7,
title = "RIDE: real-time massive image processing platform on distributed environment",
abstract = "As the demand for real-time data processing increases, a high-speed processing platform for large-scale stream data becomes necessary. For fast processing large-scale stream data, it is essential to use multiple distributed nodes. So far, there have been few studies on real-time massive image processing through efficient management and allocation of heterogeneous resources for various user-specified nodes on distributed environments. In this paper, we shall present a new platform called RIDE (Real-time massive Image processing platform on Distributed Environment) which efficiently allocates resources and executes load balancing according to the amount of stream data on distributed environments. It minimizes communication overhead by using a parallel processing strategy which handles the stream data considering both coarse-grained and fine-grained parallelism simultaneously. Coarse-grained parallelism is achieved by the automatic allocation of input streams onto partitions of broker buffer each processed by its corresponding worker node, and maximized by adaptive resource management which adjusts the number of worker nodes in a group according to the frame rate in real time. Fine-grained parallelism is achieved by parallel processing of task on each worker node and maximized by allocating heterogeneous resources such as GPU and embedded machines appropriately. Moreover, it provides a scheme of application topology which has a great advantage for higher performance by configuring the worker nodes of each stage using adaptive heterogeneous resource management. Finally, it supports dynamic fault tolerance for real-time image processing through the coordination between components in our system.",
keywords = "Distributed and parallel processing, Heterogeneous computing, Image processing, Real-time",
author = "Kim, {Yoon Ki} and Yongsung Kim and Chang-Sung Jeong",
year = "2018",
month = "12",
day = "1",
doi = "10.1186/s13640-018-0279-5",
language = "English",
volume = "2018",
journal = "Eurasip Journal on Image and Video Processing",
issn = "1687-5176",
publisher = "Springer Publishing Company",
number = "1",

}

TY - JOUR

T1 - RIDE

T2 - real-time massive image processing platform on distributed environment

AU - Kim, Yoon Ki

AU - Kim, Yongsung

AU - Jeong, Chang-Sung

PY - 2018/12/1

Y1 - 2018/12/1

N2 - As the demand for real-time data processing increases, a high-speed processing platform for large-scale stream data becomes necessary. For fast processing large-scale stream data, it is essential to use multiple distributed nodes. So far, there have been few studies on real-time massive image processing through efficient management and allocation of heterogeneous resources for various user-specified nodes on distributed environments. In this paper, we shall present a new platform called RIDE (Real-time massive Image processing platform on Distributed Environment) which efficiently allocates resources and executes load balancing according to the amount of stream data on distributed environments. It minimizes communication overhead by using a parallel processing strategy which handles the stream data considering both coarse-grained and fine-grained parallelism simultaneously. Coarse-grained parallelism is achieved by the automatic allocation of input streams onto partitions of broker buffer each processed by its corresponding worker node, and maximized by adaptive resource management which adjusts the number of worker nodes in a group according to the frame rate in real time. Fine-grained parallelism is achieved by parallel processing of task on each worker node and maximized by allocating heterogeneous resources such as GPU and embedded machines appropriately. Moreover, it provides a scheme of application topology which has a great advantage for higher performance by configuring the worker nodes of each stage using adaptive heterogeneous resource management. Finally, it supports dynamic fault tolerance for real-time image processing through the coordination between components in our system.

AB - As the demand for real-time data processing increases, a high-speed processing platform for large-scale stream data becomes necessary. For fast processing large-scale stream data, it is essential to use multiple distributed nodes. So far, there have been few studies on real-time massive image processing through efficient management and allocation of heterogeneous resources for various user-specified nodes on distributed environments. In this paper, we shall present a new platform called RIDE (Real-time massive Image processing platform on Distributed Environment) which efficiently allocates resources and executes load balancing according to the amount of stream data on distributed environments. It minimizes communication overhead by using a parallel processing strategy which handles the stream data considering both coarse-grained and fine-grained parallelism simultaneously. Coarse-grained parallelism is achieved by the automatic allocation of input streams onto partitions of broker buffer each processed by its corresponding worker node, and maximized by adaptive resource management which adjusts the number of worker nodes in a group according to the frame rate in real time. Fine-grained parallelism is achieved by parallel processing of task on each worker node and maximized by allocating heterogeneous resources such as GPU and embedded machines appropriately. Moreover, it provides a scheme of application topology which has a great advantage for higher performance by configuring the worker nodes of each stage using adaptive heterogeneous resource management. Finally, it supports dynamic fault tolerance for real-time image processing through the coordination between components in our system.

KW - Distributed and parallel processing

KW - Heterogeneous computing

KW - Image processing

KW - Real-time

UR - http://www.scopus.com/inward/record.url?scp=85048199611&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048199611&partnerID=8YFLogxK

U2 - 10.1186/s13640-018-0279-5

DO - 10.1186/s13640-018-0279-5

M3 - Article

AN - SCOPUS:85048199611

VL - 2018

JO - Eurasip Journal on Image and Video Processing

JF - Eurasip Journal on Image and Video Processing

SN - 1687-5176

IS - 1

M1 - 39

ER -