Byte-index chunking algorithm for data deduplication system

Ider Lkhagvasuren, Jung Min So, Jeong Gun Lee, Chuck Yoo, Young Woong Ko

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

This paper presents an algorithm and structure for a deduplication method which can be efficiently used for eliminating identical data between files existing different machines with high rate and performing it within rapid time. The algorithm predicts identical parts between source and destination files very fast, and then assures the identical parts and transfers only those parts of blocks that proved to be unique region. The fundamental aspect of reaching faster and high scalability determining duplicate result is that data are expressed as fixed-size block chunks which are distributed to "Index-table" by chunk's both side boundary values. "Index-table" is a fixed sized table structure; chunk's boundary byte values are used as their cell row and column numbers. Experiment result shows that the proposed solution enhances data deduplication performance and reduces data storage capacity extensively.

Original languageEnglish
Pages (from-to)415-424
Number of pages10
JournalInternational Journal of Security and its Applications
Volume7
Issue number5
DOIs
Publication statusPublished - 2013

Keywords

  • Anchor byte
  • Byte-index table
  • Chunk
  • Deduplication
  • Index-table

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Byte-index chunking algorithm for data deduplication system'. Together they form a unique fingerprint.

  • Cite this