TY - GEN
T1 - Data Transfer Advisor with Transport Profiling Optimization
AU - Yun, Daqing
AU - Wu, Chase Q.
AU - Rao, Nageswara S.V.
AU - Liu, Qiang
AU - Kettimuthu, Rajkumar
AU - Jung, Eun Sung
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/11/14
Y1 - 2017/11/14
N2 - The network infrastructures have been rapidly upgraded in many high-performance networks (HPNs). However, such infrastructure investment has not led to corresponding performance improvement in big data transfer, especially at the application layer, largely due to the complexity of optimizing transport control on end hosts. We design and implement ProbData, a PRofiling Optimization Based DAta Transfer Advisor, to help users determine the most effective data transfer method with the most appropriate control parameter values to achieve the best data transfer performance. ProbData employs a profiling optimization-based approach to exploit the optimal operational zone of various data transfer methods in support of big data transfer in extreme-scale scientific applications. We present a theoretical framework of the optimized profiling approach employed in ProbData as well as its detailed design and implementation. The advising procedure and performance benefits of ProbData are illustrated and evaluated by proof-of-concept experiments in real-life networks.
AB - The network infrastructures have been rapidly upgraded in many high-performance networks (HPNs). However, such infrastructure investment has not led to corresponding performance improvement in big data transfer, especially at the application layer, largely due to the complexity of optimizing transport control on end hosts. We design and implement ProbData, a PRofiling Optimization Based DAta Transfer Advisor, to help users determine the most effective data transfer method with the most appropriate control parameter values to achieve the best data transfer performance. ProbData employs a profiling optimization-based approach to exploit the optimal operational zone of various data transfer methods in support of big data transfer in extreme-scale scientific applications. We present a theoretical framework of the optimized profiling approach employed in ProbData as well as its detailed design and implementation. The advising procedure and performance benefits of ProbData are illustrated and evaluated by proof-of-concept experiments in real-life networks.
KW - Big data transfer
KW - data transfer advising
KW - high-performance networks
KW - profiling optimization
UR - http://www.scopus.com/inward/record.url?scp=85040590677&partnerID=8YFLogxK
U2 - 10.1109/LCN.2017.23
DO - 10.1109/LCN.2017.23
M3 - Conference contribution
AN - SCOPUS:85040590677
T3 - Proceedings - Conference on Local Computer Networks, LCN
SP - 269
EP - 277
BT - Proceedings - 2017 IEEE 42nd Conference on Local Computer Networks, LCN 2017
PB - IEEE Computer Society
T2 - 42nd IEEE Conference on Local Computer Networks, LCN 2017
Y2 - 9 October 2017 through 12 October 2017
ER -
