Multi-Dimensional Data Compression and Query Processing in Array Databases

Minsoo Kim, Hyubjin Lee, Yon Dohn Chung

Research output: Contribution to journalArticlepeer-review

Abstract

In recent times, the production of multidimensional data in various domains and their storage in array databases has witnessed a sharp increase; this rapid growth in data volumes necessitates compression in array databases. However, existing compression schemes used in array databases are general-purpose and not designed specifically for the databases. They could degrade query performance with complex analytical tasks, which incur huge computing costs. Thus, a compression scheme that considers the workflow of array databases is required. This study presents a compression scheme, SEACOW, for storing and querying multidimensional array data. The scheme is specially designed to be efficient for both dimension-based and value-based exploration. It considers data access patterns for exploration queries and embeds a synopsis, which can be utilized as an index, in the compressed array. In addition, we implement an array storage system, namely MSDB, to perform experiments. We evaluate query performance on real scientific datasets and compared it with those of existing compression schemes. Finally, our experiments demonstrate that SEACOW provides high compression rates compared to existing compression schemes, and the synopsis improves analytical query processing performance.

Original languageEnglish
Pages (from-to)111528-111544
Number of pages17
JournalIEEE Access
Volume10
DOIs
Publication statusPublished - 2022

Keywords

  • Arrays
  • Huffman coding
  • data compression
  • data structures
  • database systems
  • discrete wavelet transforms
  • indexes
  • query processing
  • scientific computing
  • tree data structures

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Multi-Dimensional Data Compression and Query Processing in Array Databases'. Together they form a unique fingerprint.

Cite this