Experimental evaluation of infiniband transport over local- and wide-area networks

Steven Carter, Makia Minich, Nageswara S.V. Rao

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations

Abstract

High-performance file systems that span across supercomputers and storage systems are becoming increasingly important, as these resources are being dispersed both within the enterprise and across the wide-area. Infiniband transport has been proposed as a candidate to provide high-performance access over interconnects primarily within an enterprise. In this paper, we present an experimental study of Infiniband transport over ORNL infrastructure of Cray XT3 supercomputer and cluster storage system for local-area and DOE UltraScience Network (USN) for wide-area. Our results indicate that Infiniband throughput of 7.6Gbps (4x) scales to 8,000 miles with 5% degradation with no customization over 10Gbps wide-area connections. We conducted preliminary performance tests of Lustre file system over these connections, which identified potential parameter and modules optimizations.

Original languageEnglish
Title of host publicationAgent Directed Simulation Symposium, ADS 2007 - Proceedings of the 2007 Spring Simulation Multiconference, SpringSim 2007
PublisherAssociation for Computing Machinery, Inc
Pages419-426
Number of pages8
ISBN (Electronic)1565553136, 9781565553132
StatePublished - Mar 25 2007
Event2007 Agent Directed Simulation Symposium, ADS 2007 - Norfolk, United States
Duration: Mar 25 2007Mar 29 2007

Publication series

NameAgent Directed Simulation Symposium, ADS 2007 - Proceedings of the 2007 Spring Simulation Multiconference, SpringSim 2007

Conference

Conference2007 Agent Directed Simulation Symposium, ADS 2007
Country/TerritoryUnited States
CityNorfolk
Period03/25/0703/29/07

Funding

This research is sponsored by the High Performance Networking Program of the Office of Science, U.S. Department of Energy, under Contract No. DE-AC05-00OR22725 with UT-Battelle, LLC, and by National Science Foundation Under Grant No. ANI-0335185.

FundersFunder number
National Science Foundation UnderANI-0335185
U.S. Department of EnergyDE-AC05-00OR22725 with UT-Battelle
Office of Science

    Keywords

    • Infiniband
    • Lustre file system
    • Storage system
    • Supercomputers
    • Wide-area connections

    Fingerprint

    Dive into the research topics of 'Experimental evaluation of infiniband transport over local- and wide-area networks'. Together they form a unique fingerprint.

    Cite this