Emulation Framework for Distributed Large-Scale Systems Integration

Neena Imam, Nageswara S.V. Rao, Anees Al-Najjar, Thomas Naughton, Seth Hitefield

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Recent trends in systems engineering include integration of very large-scale systems, which entails significant challenges when they are geographically dispersed. In these scenarios, intelligent integration of distributed large-scale systems requires significant coordination among hardware elements as well as all software components. The approach of integrated systems (both computing platform and experimental equipment) for end-to-end orchestration is called federation. Virtual frameworks can aid in the testing, assessment, and implementation of a functional system of interconnected resources. We present an emulation framework that replicates the software environments of multi-site federations of computing systems and instruments. Our emulation framework allows systems engineers to reduce development cost and avoid disruptions to production infrastructure. Our framework was effectively used to develop and test software modules for various tasks including container orchestration and instrument access. For performance assessment, however, the emulated framework is severely limited in providing accurate network and IO measurements at 10 Gbps and higher data rates. The data transfer performance profiles estimated using these emulated measurements are usually inaccurate for high bandwidth and high latency connections, since emulation does not accurately reflect the critical network transport dynamics. We utilize measurements from a physical testbed with hardware network emulators to obtain data transfer profiles that closely match the expected profiles for the emulated federations. We show the effectiveness of our approach by an illustrative example of integrated (federated) multi-site ultra large-scale systems that are connected via high speed wide area networks.

Original languageEnglish
Title of host publicationSysCon 2022 - 16th Annual IEEE International Systems Conference, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665439923
DOIs
StatePublished - 2022
Event16th Annual IEEE International Systems Conference, SysCon 2022 - Virtual, Online, Canada
Duration: Apr 25 2022May 23 2022

Publication series

NameSysCon 2022 - 16th Annual IEEE International Systems Conference, Proceedings

Conference

Conference16th Annual IEEE International Systems Conference, SysCon 2022
Country/TerritoryCanada
CityVirtual, Online
Period04/25/2205/23/22

Funding

This work is funded in part by the Laboratory Directed Research and Development Program, and also by SDN-SF and RAMSES projects, Office of Advanced Computing Research, U. S. Department of Energy (DOE), and is performed at Oak Ridge National Laboratory, managed by UT-Battelle, LLC for This work is funded in part by the Laboratory Directed Research and Development Program, and also by SDN-SF and RAMSES projects, Office of Advanced Computing Research, U. S. Department of Energy (DOE), and is performed at Oak Ridge National Laboratory, managed by UT-Battelle, LLC for DOE under Contract No. DE-ACO5-00OR22725

FundersFunder number
SDN-SF
U.S. Department of Energy
Oak Ridge National LaboratoryDE-ACO5-00OR22725
Laboratory Directed Research and Development
UT-Battelle
Office of Advanced Research Computing, Rutgers, The State University of New Jersey

    Keywords

    • federated instruments
    • instrument control
    • instrument steering
    • network analytics.
    • network virtualization
    • software-defined infrastructures

    Fingerprint

    Dive into the research topics of 'Emulation Framework for Distributed Large-Scale Systems Integration'. Together they form a unique fingerprint.

    Cite this