TY - GEN
T1 - Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections
AU - Rao, Nageswara S.V.
AU - Liu, Qiang
AU - Sen, Satyabrata
AU - Hinkel, Greg
AU - Imam, Neena
AU - Foster, Ian
AU - Kettimuthu, Rajkumar
AU - Settlemyer, Bradley W.
AU - Wu, Chase Q.
AU - Yun, Daqing
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2017/1/20
Y1 - 2017/1/20
N2 - File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.
AB - File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.
KW - Dedicated connections
KW - File I/O read and write
KW - Profiling
KW - Profiling overhead
KW - RTT
KW - TCP
KW - Throughput
KW - Wide area transport
UR - https://www.scopus.com/pages/publications/85013635220
U2 - 10.1109/HPCC-SmartCity-DSS.2016.0038
DO - 10.1109/HPCC-SmartCity-DSS.2016.0038
M3 - Conference contribution
AN - SCOPUS:85013635220
T3 - Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
SP - 198
EP - 205
BT - Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
A2 - Yang, Laurence T.
A2 - Chen, Jinjun
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
Y2 - 12 December 2016 through 14 December 2016
ER -