Abstract
Specialized, transient data services are playing an increasingly prominent role in data-intensive scientific computing. These services offer flexible, on-demand pairing of applications with storage hardware using semantics that are optimized for the problem domain. Concurrent with this trend, upcoming scientific computing and big data systems will be deployed with emerging non-volatile memory (NVM) technology to achieve the highest possible price/productivity ratio. Clearly, therefore, we must develop techniques to facilitate the confluence of specialized data services and NVM technology. In this work we explore how to enable the composition of NVM resources within transient distributed services while still retaining their essential performance characteristics. Our approach involves eschewing the conventional shared file system model and instead projecting NVM devices as remote microservices that leverage user-level threads, remote procedure call (RPC) services, remote direct memory access (RDMA) enabled network transports, and persistent memory libraries in order to maximize performance. We describe a prototype system that incorporates these concepts, evaluate its performance for key workloads on an exemplar system, and discuss how the system can be leveraged as a component of future data-intensive architectures.
| Original language | English |
|---|---|
| State | Published - 2016 |
| Event | 4th Workshop on Interactions of NVM/Flash with Operating Systems and Workloads, INFLOW 2016, co-located with OSDI 2016 - Savannah, United States Duration: Nov 1 2016 → … |
Conference
| Conference | 4th Workshop on Interactions of NVM/Flash with Operating Systems and Workloads, INFLOW 2016, co-located with OSDI 2016 |
|---|---|
| Country/Territory | United States |
| City | Savannah |
| Period | 11/1/16 → … |
Funding
This work was supported by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research, under Contract DE-AC02-06CH11357. This research used resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357. This research also used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.
| Funders | Funder number |
|---|---|
| DOE Office of Science | DE-AC05-00OR22725 |
| U.S. Department of Energy | |
| Office of Science | |
| Advanced Scientific Computing Research | DE-AC02-06CH11357 |