Improving HDFS performance using local caching system

Sang Deok Yoon, In Yang Jung, Ki Hyun Kim, Chang-Sung Jeong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

Original languageEnglish
Title of host publication2nd International Conference on Future Generation Communication Technologies, FGCT 2013
PublisherIEEE Computer Society
Pages153-156
Number of pages4
DOIs
Publication statusPublished - 2013 Jan 1
Event2nd International Conference on Future Generation Communication Technologies, FGCT 2013 - London, United Kingdom
Duration: 2013 Nov 122013 Nov 14

Other

Other2nd International Conference on Future Generation Communication Technologies, FGCT 2013
CountryUnited Kingdom
CityLondon
Period13/11/1213/11/14

Keywords

  • cache
  • distributed file system
  • hadoop
  • HDFS
  • locality

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Yoon, S. D., Jung, I. Y., Kim, K. H., & Jeong, C-S. (2013). Improving HDFS performance using local caching system. In 2nd International Conference on Future Generation Communication Technologies, FGCT 2013 (pp. 153-156). [6767200] IEEE Computer Society. https://doi.org/10.1109/FGCT.2013.6767200

Improving HDFS performance using local caching system. / Yoon, Sang Deok; Jung, In Yang; Kim, Ki Hyun; Jeong, Chang-Sung.

2nd International Conference on Future Generation Communication Technologies, FGCT 2013. IEEE Computer Society, 2013. p. 153-156 6767200.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yoon, SD, Jung, IY, Kim, KH & Jeong, C-S 2013, Improving HDFS performance using local caching system. in 2nd International Conference on Future Generation Communication Technologies, FGCT 2013., 6767200, IEEE Computer Society, pp. 153-156, 2nd International Conference on Future Generation Communication Technologies, FGCT 2013, London, United Kingdom, 13/11/12. https://doi.org/10.1109/FGCT.2013.6767200
Yoon SD, Jung IY, Kim KH, Jeong C-S. Improving HDFS performance using local caching system. In 2nd International Conference on Future Generation Communication Technologies, FGCT 2013. IEEE Computer Society. 2013. p. 153-156. 6767200 https://doi.org/10.1109/FGCT.2013.6767200
Yoon, Sang Deok ; Jung, In Yang ; Kim, Ki Hyun ; Jeong, Chang-Sung. / Improving HDFS performance using local caching system. 2nd International Conference on Future Generation Communication Technologies, FGCT 2013. IEEE Computer Society, 2013. pp. 153-156
@inproceedings{5a60ef67721643928769c18f731c31f6,
title = "Improving HDFS performance using local caching system",
abstract = "I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.",
keywords = "cache, distributed file system, hadoop, HDFS, locality",
author = "Yoon, {Sang Deok} and Jung, {In Yang} and Kim, {Ki Hyun} and Chang-Sung Jeong",
year = "2013",
month = "1",
day = "1",
doi = "10.1109/FGCT.2013.6767200",
language = "English",
pages = "153--156",
booktitle = "2nd International Conference on Future Generation Communication Technologies, FGCT 2013",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Improving HDFS performance using local caching system

AU - Yoon, Sang Deok

AU - Jung, In Yang

AU - Kim, Ki Hyun

AU - Jeong, Chang-Sung

PY - 2013/1/1

Y1 - 2013/1/1

N2 - I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

AB - I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.

KW - cache

KW - distributed file system

KW - hadoop

KW - HDFS

KW - locality

UR - http://www.scopus.com/inward/record.url?scp=84898969542&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898969542&partnerID=8YFLogxK

U2 - 10.1109/FGCT.2013.6767200

DO - 10.1109/FGCT.2013.6767200

M3 - Conference contribution

AN - SCOPUS:84898969542

SP - 153

EP - 156

BT - 2nd International Conference on Future Generation Communication Technologies, FGCT 2013

PB - IEEE Computer Society

ER -