Performance optimization of a distributed transcoding system based on Hadoop for multimedia streaming services

Myoungjin Kim, Seungho Han, Yun Cui, Hanku Lee, Chang-Sung Jeong

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

In recent times, significant progress has been achieved in cost-effective and timely processing of large amounts of data through Hadoop based on the emerging MapReduce framework. Based on these developments, we proposed a Hadoop-based Distributed Video Transcoding System which transcodes large video data sets into specific video formats depending on user-requested options. In order to reduce the transcoding time exponentially, we apply a Hadoop Distributed File System and a MapReduce framework to our system. Hadoop and MapReduce are designed to process petabyte-scale text data in a parallel and distributed manner. However, our system processes multi-media data. In this study, we measure the total transcoding time for various values of five MapReduce tuning parameters: block replication factor, Hadoop Distributed File System block size, Java Virtual Machine reuse option, maximum number of map slots and input/output buffer size. Thus, based on the experimental results, we determine the optimal values of the parameters affecting transcoding processing in order to improve the performance of our Hadoop-based system that processes a large amount of video data. From the results, it is clearly observed that our system exhibits a notable difference in transcoding performance depending on the values of the MapReduce tuning parameters.

Original languageEnglish
Pages (from-to)2099-2109
Number of pages11
JournalInformation (Japan)
Volume18
Issue number5
Publication statusPublished - 2015 May 1

Fingerprint

Tuning
Processing
Costs
Virtual machine

Keywords

  • And cloud computing
  • Hadoop Optimization
  • MapReduce
  • Performance tuning
  • Video transcoding system

ASJC Scopus subject areas

  • General

Cite this

Performance optimization of a distributed transcoding system based on Hadoop for multimedia streaming services. / Kim, Myoungjin; Han, Seungho; Cui, Yun; Lee, Hanku; Jeong, Chang-Sung.

In: Information (Japan), Vol. 18, No. 5, 01.05.2015, p. 2099-2109.

Research output: Contribution to journalArticle

Kim, Myoungjin ; Han, Seungho ; Cui, Yun ; Lee, Hanku ; Jeong, Chang-Sung. / Performance optimization of a distributed transcoding system based on Hadoop for multimedia streaming services. In: Information (Japan). 2015 ; Vol. 18, No. 5. pp. 2099-2109.
@article{2c2c3ff9801c48659220458f3cde6e5e,
title = "Performance optimization of a distributed transcoding system based on Hadoop for multimedia streaming services",
abstract = "In recent times, significant progress has been achieved in cost-effective and timely processing of large amounts of data through Hadoop based on the emerging MapReduce framework. Based on these developments, we proposed a Hadoop-based Distributed Video Transcoding System which transcodes large video data sets into specific video formats depending on user-requested options. In order to reduce the transcoding time exponentially, we apply a Hadoop Distributed File System and a MapReduce framework to our system. Hadoop and MapReduce are designed to process petabyte-scale text data in a parallel and distributed manner. However, our system processes multi-media data. In this study, we measure the total transcoding time for various values of five MapReduce tuning parameters: block replication factor, Hadoop Distributed File System block size, Java Virtual Machine reuse option, maximum number of map slots and input/output buffer size. Thus, based on the experimental results, we determine the optimal values of the parameters affecting transcoding processing in order to improve the performance of our Hadoop-based system that processes a large amount of video data. From the results, it is clearly observed that our system exhibits a notable difference in transcoding performance depending on the values of the MapReduce tuning parameters.",
keywords = "And cloud computing, Hadoop Optimization, MapReduce, Performance tuning, Video transcoding system",
author = "Myoungjin Kim and Seungho Han and Yun Cui and Hanku Lee and Chang-Sung Jeong",
year = "2015",
month = "5",
day = "1",
language = "English",
volume = "18",
pages = "2099--2109",
journal = "Information (Japan)",
issn = "1343-4500",
publisher = "International Information Institute",
number = "5",

}

TY - JOUR

T1 - Performance optimization of a distributed transcoding system based on Hadoop for multimedia streaming services

AU - Kim, Myoungjin

AU - Han, Seungho

AU - Cui, Yun

AU - Lee, Hanku

AU - Jeong, Chang-Sung

PY - 2015/5/1

Y1 - 2015/5/1

N2 - In recent times, significant progress has been achieved in cost-effective and timely processing of large amounts of data through Hadoop based on the emerging MapReduce framework. Based on these developments, we proposed a Hadoop-based Distributed Video Transcoding System which transcodes large video data sets into specific video formats depending on user-requested options. In order to reduce the transcoding time exponentially, we apply a Hadoop Distributed File System and a MapReduce framework to our system. Hadoop and MapReduce are designed to process petabyte-scale text data in a parallel and distributed manner. However, our system processes multi-media data. In this study, we measure the total transcoding time for various values of five MapReduce tuning parameters: block replication factor, Hadoop Distributed File System block size, Java Virtual Machine reuse option, maximum number of map slots and input/output buffer size. Thus, based on the experimental results, we determine the optimal values of the parameters affecting transcoding processing in order to improve the performance of our Hadoop-based system that processes a large amount of video data. From the results, it is clearly observed that our system exhibits a notable difference in transcoding performance depending on the values of the MapReduce tuning parameters.

AB - In recent times, significant progress has been achieved in cost-effective and timely processing of large amounts of data through Hadoop based on the emerging MapReduce framework. Based on these developments, we proposed a Hadoop-based Distributed Video Transcoding System which transcodes large video data sets into specific video formats depending on user-requested options. In order to reduce the transcoding time exponentially, we apply a Hadoop Distributed File System and a MapReduce framework to our system. Hadoop and MapReduce are designed to process petabyte-scale text data in a parallel and distributed manner. However, our system processes multi-media data. In this study, we measure the total transcoding time for various values of five MapReduce tuning parameters: block replication factor, Hadoop Distributed File System block size, Java Virtual Machine reuse option, maximum number of map slots and input/output buffer size. Thus, based on the experimental results, we determine the optimal values of the parameters affecting transcoding processing in order to improve the performance of our Hadoop-based system that processes a large amount of video data. From the results, it is clearly observed that our system exhibits a notable difference in transcoding performance depending on the values of the MapReduce tuning parameters.

KW - And cloud computing

KW - Hadoop Optimization

KW - MapReduce

KW - Performance tuning

KW - Video transcoding system

UR - http://www.scopus.com/inward/record.url?scp=84939246160&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84939246160&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:84939246160

VL - 18

SP - 2099

EP - 2109

JO - Information (Japan)

JF - Information (Japan)

SN - 1343-4500

IS - 5

ER -