MARS: A multi-level array representation for simulation data

Minsoo Kim, Ilhyun Suh, Yon Dohn Chung

Research output: Contribution to journalArticle

Abstract

In the numerical simulation domain, owing to the huge size of data and the complexity of implementing the domain specific applications, a database-centric approach for handling multidimensional simulation data is gaining considerable attention. Array databases provide an optimized set of features to support administrating multidimensional data; representing simulation data with an array can be an optimal choice. Generally, query performance on sparsely filled arrays, especially when empty cells are placed between adjacent elements, can be poor. In this context, previous studies focused on the compact representation of simulation data by reducing the number of empty cells between adjacent elements as possible. However, these methods inevitably lose the original spatial structure of elements (i.e., the relative distance and direction among elements), making it impossible to utilize the built-in multidimensional operators provided by array databases. In this paper, we propose MARS, a multi-level array representation for simulation data. MARS utilizes multiple level arrays with various resolutions to cope with the two addressed problems. In the MARS representation, elements tend to be concentrated into dense array regions, where each region is selectively stored in one of the level arrays that most reduces the empty cells between adjacent elements. Unlike existing methods, MARS retains the spatial structure of elements, and thus no additional efforts to reorganize the original spatial structure for query processing is required. We built MARS on top of SciDB and implemented a specialized command line tool for MARS. We present methods and optimized operators for query processing over MARS. We evaluate the performance of MARS using two real-world numerical simulation datasets.

Original languageEnglish
Pages (from-to)419-434
Number of pages16
JournalFuture Generation Computer Systems
Volume111
DOIs
Publication statusAccepted/In press - 2019 Jan 1

Keywords

  • Array databases
  • Query processing
  • Scientific data

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'MARS: A multi-level array representation for simulation data'. Together they form a unique fingerprint.

  • Cite this