2PTS: A two-phase task scheduling algorithm for MapReduce

Byungnam Lim, Yeeun Shim, Yon Dohn Chung

Research output: Contribution to journalArticle

Abstract

For an efficient processing of large data in a distributed system, Hadoop MapReduce performs task scheduling such that tasks are distributed with consideration of the data locality. The data locality, however, is limitedly exploited, since it is pursued one node at a time basis without considering the global optimality. In this paper, we propose a novel task scheduling algorithm that globally considers the data locality. Through experiments, we show our algorithm improves the performance of MapReduce in various situations.

Original languageEnglish
Pages (from-to)2377-2380
Number of pages4
JournalIEICE Transactions on Information and Systems
VolumeE99D
Issue number9
DOIs
Publication statusPublished - 2016 Sep 1

Keywords

  • Data locality
  • MapReduce
  • Task scheduling algorithm

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of '2PTS: A two-phase task scheduling algorithm for MapReduce'. Together they form a unique fingerprint.

  • Cite this