MOSIQS: Persistent Memory Object Storage with Metadata Indexing and Querying for Scientific Computing

Awais Khan, Hyogi Sim, Sudharshan S. Vazhkudai, Youngjae Kim

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Scientific applications often require high-bandwidth shared storage to perform joint simulations and collaborative data analytics. Shared memory pools provide a chance to satisfy such needs. Recently, a high-speed network such as Gen-Z utilizing persistent memory (PM) offers an opportunity to create a shared memory pool connected to compute nodes. However, there are several challenges to use scientific applications on the shared memory pool directly such as scalability, failure-atomicity, and lack of scientific metadata-based search and query. In this paper, we propose MOSIQS, a persistent memory object storage framework with metadata indexing and querying for scientific computing. We design MOSIQS based on the key idea that memory objects on PM pool can live beyond the application lifetime and can become the sharing currency for applications and scientists. MOSIQS provides an aggregate memory pool atop an array of persistent memory devices to store and access memory objects to accelerate scientific computing. MOSIQS uses a lightweight persistent memory key-value store to manage the metadata of memory objects, which enables memory object sharing. To facilitate metadata search and query over millions of memory objects resident on memory pool, we introduce Group Split and Merge (GSM), a novel persistent index data structure designed primarily for scientific datasets. GSM splits and merges dynamically to minimize the query search space and maintains low query processing time while overcoming the index storage overhead. MOSIQS is implemented on top of PMDK. We evaluate the proposed approach on many-core server with an array of real PM devices. Experimental results show that MOSIQS gains a 100% write performance improvement and executes multi-attribute queries efficiently with 2.7 × less index storage overhead offering significant potential to speed up scientific computing applications.

Original languageEnglish
Article number9448213
Pages (from-to)85217-85231
Number of pages15
JournalIEEE Access
Volume9
DOIs
StatePublished - 2021
Externally publishedYes

Funding

This work was supported in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) Grant funded by the Korean Government [Ministry of Science and ICT (MSIT)] (Researches on Next Generation Memory-Centric Computing System Architecture) under Grant 2018-0-00503, and in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant NRF-2021R1A2C2014386.

Keywords

  • Memory-centric computing and HPC
  • PM index data structures
  • persistent memory storage
  • scientific metadata indexing and search

Fingerprint

Dive into the research topics of 'MOSIQS: Persistent Memory Object Storage with Metadata Indexing and Querying for Scientific Computing'. Together they form a unique fingerprint.

Cite this