Abstract
The increase of computer processing speed is significantly outpacing improvements in network and storage bandwidth, leading to the big data challenge in modern science, where scientific applications can quickly generate much more data than that can be transferred and stored. As a result, big scientific data must be reduced by a few orders of magnitude while the accuracy of the reduced data needs to be guaranteed for further scientific explorations. Moreover, scientists are often interested in some specific spatial/temporal regions in their data, where higher accuracy is required. The locations of the regions requiring high accuracy can sometimes be prescribed based on application knowledge, while other times they must be estimated based on general spatial/temporal variation. In this paper, we develop a novel multilevel approach which allows users to impose region-wise compression error bounds. Our method utilizes the byproduct of a multilevel compressor to detect regions where details are rich and we provide the theoretical underpinning for region-wise error control. With spatially varying precision preservation, our approach can achieve significantly higher compression ratios than single-error bounded compression approaches and control errors in the regions of interest. We conduct the evaluations on two climate use cases-one targeting small-scale, node features and the other focusing on long, areal features. For both use cases, the locations of the features were unknown ahead of the compression. By selecting approximately 16% of the data based on multi-scale spatial variations and compressing those regions with smaller error tolerances than the rest, our approach improves the accuracy of post-analysis by approximately 2 × compared to single-error-bounded compression at the same compression ratio. Using the same error bound for the region of interest, our approach can achieve an increase of more than 50% in overall compression ratio.
Original language | English |
---|---|
Title of host publication | Scientific and Statistical Database Management - 34th International Conference, SSDBM 2022 - Proceedings |
Editors | Elaheh Pourabbas, Yongluan Zhou, Yuchen Li, Bin Yang |
Publisher | Association for Computing Machinery |
ISBN (Electronic) | 9781450396677 |
DOIs | |
State | Published - Jul 6 2022 |
Event | 34th International Conference on Scientific and Statistical Database Management, SSDBM 2022 - Copenhagen, Denmark Duration: Jul 6 2022 → Jul 8 2022 |
Publication series
Name | ACM International Conference Proceeding Series |
---|
Conference
Conference | 34th International Conference on Scientific and Statistical Database Management, SSDBM 2022 |
---|---|
Country/Territory | Denmark |
City | Copenhagen |
Period | 07/6/22 → 07/8/22 |
Funding
project through DRD program of Oak Ridge National Laboratory. The climate simulation data were obtained from the Energy Exas-cale Earth System Model project, sponsored by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research. This research was supported by the ECP CODAR, Sirius-2, and RAPIDS-2 projects through the Advanced Scientific Computing Research (ASCR) program of Department of Energy, and the LDRD This research was supported by the ECP CODAR, Sirius-2, and RAPIDS-2 projects through the Advanced Scientific Computing Research (ASCR) program of Department of Energy, and the LDRD project through DRD program of Oak Ridge National Laboratory. The climate simulation data were obtained from the Energy Exascale Earth System Model project, sponsored by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research.
Keywords
- Climate Data Compression
- Error Control
- Region-adaptive Lossy Compression