High-speed dedicated channels and experimental results with Hurricane protocol

Networks are currently being deployed to provide dedicated channels to support large data transfers and stable control flow needed in large-scale scientific applications. We present experimental results on application-level throughputs achievable on such channels using a range of hosts and dedicated connections. These results high-light the throughput limitations in several cases due to host issues, including disk and file system speeds, processor scheduling and loads, and complexity of internal data paths. We characterize such effects using the notion of host-bandwidth, which must be considered together with the connection-bandwidth in designing and optimizing transport protocols for dedicated channels. We propose a new transport protocol implementation, named Hurricane, to achieve high utilization of dedicated channels. While the overall protocol is quite similar to existing UDP-based protocols, new parameters, such as group size of NACKS, are identified and carefully optimized to achieve high channel utilization. Our end hosts consist of workstations, a cluster and Cray X1 supercomputer. Between two workstations, we consider: (A) 1 Gbps layer 3 connection of several hundred miles, and (B) 10 Gbps layer 2 connection of several thousand miles. Between Cray X1 and the cluster, we consider: (C) 450 Mbps layer 3 channel provisioned by policy, and (D) 1 Gbps layer 2 connection provisioned over an MPLS tunnel.