Abstract
Recent developments in software-defined infrastructures promise that scientific workflows utilizing supercomputers, instruments, and storage systems will be dynamically composed and orchestrated using software at unprecedented speed and scale in the near future. Testing of the underlying networking software, particularly during initial exploratory stages, remains a challenge due to potential disruptions, and resource allocation and coordination needed over the multi-domain physical infrastructure. To overcome these challenges, we develop the Virtual Science Network Environment (VSNE) that emulates the multi-site host, storage, and network infrastructure using Virtual Machines (VMs), wherein the production and nascent software can be tested. Within each VM, which represents a site, the hosts and local-area networks are emulated using Mininet, and the Software-Defined Network (SDN) controllers and service daemon codes are natively run to support dynamic provisioning of network connections. Additionally, Lustre filesystem support at the sites and an emulation of the long-haul network using Mininet, are provided using separate VMs. As case studies, we describe Lustre file transfers using XDD, Red5 streaming service demonstration, and an emulated experiment with remote monitoring and steering modules, all supported over dynamically configured connections using SDN controllers.
Original language | English |
---|---|
Title of host publication | Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 - In conjunction with HPDC |
Publisher | Association for Computing Machinery, Inc |
ISBN (Electronic) | 9781450358620 |
DOIs | |
State | Published - Jun 11 2018 |
Event | 1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 - Tempe, United States Duration: Jun 11 2018 → … |
Publication series
Name | Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 - In conjunction with HPDC |
---|
Conference
Conference | 1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 |
---|---|
Country/Territory | United States |
City | Tempe |
Period | 06/11/18 → … |
Funding
This work is funded by the High-Performance Networking Program, Office of Advanced Computing Research, U.S. Department of Energy, and by Extreme Scale Systems Center, sponsored by U. S. Department of Defense, and performed at Oak Ridge National Laboratory managed by UT-Battelle, LLC for U.S. Department of Energy under Contract No. DE-AC05-00OR22725.
Keywords
- Scientific workflows
- Software-defined infrastructure
- Software-defined networking
- Virtual science network environment