Abstract
Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.
| Original language | English |
|---|---|
| Title of host publication | 2020 IEEE International Conference on Communications, ICC 2020 - Proceedings |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| ISBN (Electronic) | 9781728150895 |
| DOIs | |
| State | Published - Jun 2020 |
| Event | 2020 IEEE International Conference on Communications, ICC 2020 - Dublin, Ireland Duration: Jun 7 2020 → Jun 11 2020 |
Publication series
| Name | IEEE International Conference on Communications |
|---|---|
| Volume | 2020-June |
| ISSN (Print) | 1550-3607 |
Conference
| Conference | 2020 IEEE International Conference on Communications, ICC 2020 |
|---|---|
| Country/Territory | Ireland |
| City | Dublin |
| Period | 06/7/20 → 06/11/20 |
Funding
ACKNOWLEDGMENTS This research is sponsored by U.S. National Science Foundation under Grant No. CNS-1828123 with New Jersey Institute of Technology and the Presidential Research Grant of Harrisburg University. REFERENCES [1] Energy Sciences Network. http://www.es.net. [2] ESnet OSCARS Service. https://bit.ly/2Ou9qVe. [3] Globus Data Transfer. https://bit.ly/2m10qLI. [4] Internet2 AL2S. https://goo.gl/4iAbQn. [5] Science DMZ DTNs. https://bit.ly/2k3qBQM. [6] UDT-Powered Projects. https://bit.ly/2JZtA7n. [7] D. Yun et al. Profiling optimization for big data transfer over dedicated channels. In Proc. of ICCCN ’16. [8] F. Pedregosa et al. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830, 2011. [9] Y. Gu and R. Grossman. UDT: UDP-based data transfer for high-speed wide area networks. Computer Networks, 51(7):1777–1799, 2007. [10] S. Ha, I. Rhee, and L. Xu. CUBIC: A new TCP-friendly high-speed TCP variant. ACM SIGOPS Oper. Syst. Rev., 42(5):64–74, 2008. [11] Z. Liu, P. Balaprakash, R. Kettimuthu, and I. Foster. Explaining wide area data transfer performance. In Proc. of HPDC ’17, pages 167–178. [12] Z. Liu, R. Kettimuthu, P. Balaprakash, and I. Foster. Building a wide-area data transfer performance predictor: An empirical study. In Proc. of the 1st Int’l Conf. on Machine Learning for Netw., 2018. [13] M. Ester et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. of KDD ’96, pages 226–231. [14] M. Mirza, J. Sommers, P. Barford, and X. Zhu. A machine learning ap-proach to TCP throughput prediction. IEEE/ACM Trans. on Networking, 18(4):1026–1039, 2010. [15] M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning. The MIT Press, 2nd edition, 2018. [16] N.S.V. Rao et al. TCP throughput profiles using measurements over dedicated connections. In Proc. of HPDC ’17, pages 193–204. [17] N.S.V. Rao et al. Learning concave-convex profiles of data transport over dedicated connections. In Proc. of the 1st Int’l Conf. on Machine Learning for Netw., 2018. [18] P. Ye et al. Customized regression model for Airbnb dynamic pricing. In Proc. of KDD ’18, pages 932–940. [19] Q. Liu et al. Measurement-based performance profiles and dynamics of UDT over dedicated connections. In Proc. of ICNP ’16. [20] S. Jain et al. B4: Experience with a globally-deployed software defined WAN. ACM SIGCOMM Comput. Commun. Rev., 43(4):3–14, 2013. [21] K. Winstein and H. Balakrishnan. TCP ex machina: Computer-generated congestion control. ACM SIGCOMM Comput. Commun. Rev., 43(4):123–134, 2013. This research is sponsored by U.S. National Science Foundation under Grant No. CNS-1828123
Keywords
- Performance prediction
- big data transfer
- high-performance networks
- latent variables
- machine learning