Abstract
We present a multilevel technique for the compression and reduction of univariate data and give an optimal complexity algorithm for its implementation. A hierarchical scheme offers the flexibility to produce multiple levels of partial decompression of the data so that each user can work with a reduced representation that requires minimal storage whilst achieving the required level of tolerance. The algorithm is applied to the case of turbulence modelling in which the datasets are traditionally not only extremely large but inherently non-smooth and, as such, rather resistant to compression. We decompress the data for a range of relative errors, carry out the usual analysis procedures for turbulent data, and compare the results of the analysis on the reduced datasets to the results that would be obtained on the full dataset. The results obtained demonstrate the promise of multilevel compression techniques for the reduction of data arising from large scale simulations of complex phenomena such as turbulence modelling.
| Original language | English |
|---|---|
| Pages (from-to) | 65-76 |
| Number of pages | 12 |
| Journal | Computing and Visualization in Science |
| Volume | 19 |
| Issue number | 5-6 |
| DOIs | |
| State | Published - Dec 15 2018 |
Funding
This research was supported in part by the Exascale Computing Project (17-SC-20-SC) of the U.S. Department of Energy; the Advanced Scientific Research Office (ASCR) at the Department of Energy, under contract DE-AC02-06CH11357; the DOE Storage Systems and Input/Output for Extreme Scale Science project, announcement number LAB 15-1338; and DOE and UT–Battelle, LLC, Contract Number DE-AC05-00OR22725.
| Funders | Funder number |
|---|---|
| Advanced Scientific Research Office | |
| U.S. Department of Energy | DE-AC02-06CH11357 |
| Battelle | DE-AC05-00OR22725 |
Keywords
- Data compression
- Data reduction
- Error-controlled compression
- Lossy compression
- Multilevel compression