DADS: A dynamic and adaptive data space for interacting parallel applications

Emerging scientific and engineering simulations are composed of multiple loosely coupled parallel application components that interact by sharing data at runtime. As a result, coupling and data sharing substrates that can facilitate such data exchange and asynchronous interaction between independent parallel application components, are critical. However, in real-world coupled simulation workflows, data exchange and interaction are dynamic, both in the type of interactions as well as the amount of data exchanged. As a result, the data sharing substrate must handle dynamic and changing coupling/data-exchange requirements from the applications. In this paper, we present the dynamic and adaptive data space(DADS) for coupled simulation workflows. DADS implements a scalable, distributed and dynamic in-memory data space, which is semantically specialized for simulations where data is associated with the discretization of an application domain, e.g., a grid or a mesh. DADS is dynamic in that the data space can be created and deleted at runtime to support dynamically occurring interaction and data exchange. Furthermore, a DADS space can automatically resize itself based on the data exchange requirements as well as the status of the underlying resources. DADS has been implemented and deployed on the Eugene IBM BlueGene/P, and evaluation of its performance on this system is presented.