Virtual environment for testing software-defined networking solutions for scientific workflows

Qiang Liu, Nageswara S.V. Rao, Satyabrata Sen, Joshua M. Boley, Rajkumar Kettimuthu, Bradley W. Settlemyer, Hsing Bung Chen, Dimitrios Katramatos

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Recent developments in software-defined infrastructures promise that scientific workflows utilizing supercomputers, instruments, and storage systems will be dynamically composed and orchestrated using software at unprecedented speed and scale in the near future. Testing of the underlying networking software, particularly during initial exploratory stages, remains a challenge due to potential disruptions, and resource allocation and coordination needed over the multi-domain physical infrastructure. To overcome these challenges, we develop the Virtual Science Network Environment (VSNE) that emulates the multi-site host, storage, and network infrastructure using Virtual Machines (VMs), wherein the production and nascent software can be tested. Within each VM, which represents a site, the hosts and local-area networks are emulated using Mininet, and the Software-Defined Network (SDN) controllers and service daemon codes are natively run to support dynamic provisioning of network connections. Additionally, Lustre filesystem support at the sites and an emulation of the long-haul network using Mininet, are provided using separate VMs. As case studies, we describe Lustre file transfers using XDD, Red5 streaming service demonstration, and an emulated experiment with remote monitoring and steering modules, all supported over dynamically configured connections using SDN controllers.

Original languageEnglish
Title of host publicationProceedings of the 1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 - In conjunction with HPDC
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450358620
DOIs
StatePublished - Jun 11 2018
Event1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 - Tempe, United States
Duration: Jun 11 2018 → …

Publication series

NameProceedings of the 1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018 - In conjunction with HPDC

Conference

Conference1st International Workshop on Autonomous Infrastructure for Science, AI-Science 2018
Country/TerritoryUnited States
CityTempe
Period06/11/18 → …

Funding

This work is funded by the High-Performance Networking Program, Office of Advanced Computing Research, U.S. Department of Energy, and by Extreme Scale Systems Center, sponsored by U. S. Department of Defense, and performed at Oak Ridge National Laboratory managed by UT-Battelle, LLC for U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Keywords

  • Scientific workflows
  • Software-defined infrastructure
  • Software-defined networking
  • Virtual science network environment

Fingerprint

Dive into the research topics of 'Virtual environment for testing software-defined networking solutions for scientific workflows'. Together they form a unique fingerprint.

Cite this