Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses

Nageswara S.V. Rao; Neena Imam; Zhengchun Liu; Rajkumar Kettimuthu; Ian Foster

doi:10.1007/978-3-030-45778-5_11

Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses

Nageswara S.V. Rao, Neena Imam, Zhengchun Liu, Rajkumar Kettimuthu, Ian Foster

Autonomous and Complex Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Scientific computations are expected to be increasingly distributed across wide-area networks, and Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. Application-level measurements of MPI operations reflect the connection Round-Trip Time (RTT) and loss rate, and machine learning methods have been previously developed to estimate them under deterministic periodic losses. In this paper, we consider more complex, random losses with uniform, Poisson and Gaussian distributions. We study five disparate machine leaning methods, with linear and non-linear, and smooth and non-smooth properties, to estimate RTT and loss rate over 10 Gbps connections with 0–366 ms RTT. The diversity and complexity of these estimators combined with the randomness of losses and TCP’s non-linear response together rule out the selection of a single best among them; instead, we fuse them to retain their design diversity. Overall, the results show that accurate estimates can be generated at low loss rates but become inaccurate at loss rates 10% and higher, thereby illustrating both their strengths and limitations.

Original language	English
Title of host publication	Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers
Editors	Selma Boumerdassi, Éric Renault, Paul Mühlethaler
Publisher	Springer
Pages	154-174
Number of pages	21
ISBN (Print)	9783030457778
DOIs	https://doi.org/10.1007/978-3-030-45778-5_11
State	Published - 2020
Event	2nd International Conference on Machine Learning for Networking, MLN 2019 - Paris, France Duration: Dec 3 2019 → Dec 5 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12081 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2nd International Conference on Machine Learning for Networking, MLN 2019
Country/Territory	France
City	Paris
Period	12/3/19 → 12/5/19

Funding

This work is funded by RAMSES project and Applied Mathematics program, Office of Advanced Computing Research, U.S. Department of Energy, and by Extreme Scale Systems Center, sponsored by U.S. Department of Defense, and performed at Oak Ridge National Laboratory managed by UT-Battelle, LLC for U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Funders	Funder number
Office of Advanced Computing Research
UT-Battelle
U.S. Department of Defense
U.S. Department of Energy
Oak Ridge National Laboratory
UT-Battelle	DE-AC05-00OR22725

Keywords

Generalization bounds
Information fusion
Loss rate
Machine Learning
Message Passing Interface
Regression
Round Trip Time

Access to Document

10.1007/978-3-030-45778-5_11

Cite this

Rao, N. S. V., Imam, N., Liu, Z., Kettimuthu, R., & Foster, I. (2020). Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses. In S. Boumerdassi, É. Renault, & P. Mühlethaler (Eds.), Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers (pp. 154-174). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12081 LNCS). Springer. https://doi.org/10.1007/978-3-030-45778-5_11

Rao, Nageswara S.V. ; Imam, Neena ; Liu, Zhengchun et al. / Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses. Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers. editor / Selma Boumerdassi ; Éric Renault ; Paul Mühlethaler. Springer, 2020. pp. 154-174 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ddd1859a13a1494a93d6bf802e98bf60,

title = "Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses",

abstract = "Scientific computations are expected to be increasingly distributed across wide-area networks, and Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. Application-level measurements of MPI operations reflect the connection Round-Trip Time (RTT) and loss rate, and machine learning methods have been previously developed to estimate them under deterministic periodic losses. In this paper, we consider more complex, random losses with uniform, Poisson and Gaussian distributions. We study five disparate machine leaning methods, with linear and non-linear, and smooth and non-smooth properties, to estimate RTT and loss rate over 10 Gbps connections with 0–366 ms RTT. The diversity and complexity of these estimators combined with the randomness of losses and TCP{\textquoteright}s non-linear response together rule out the selection of a single best among them; instead, we fuse them to retain their design diversity. Overall, the results show that accurate estimates can be generated at low loss rates but become inaccurate at loss rates 10% and higher, thereby illustrating both their strengths and limitations.",

keywords = "Generalization bounds, Information fusion, Loss rate, Machine Learning, Message Passing Interface, Regression, Round Trip Time",

author = "Rao, {Nageswara S.V.} and Neena Imam and Zhengchun Liu and Rajkumar Kettimuthu and Ian Foster",

note = "Publisher Copyright: {\textcopyright} 2020, IFIP International Federation for Information Processing.; 2nd International Conference on Machine Learning for Networking, MLN 2019 ; Conference date: 03-12-2019 Through 05-12-2019",

year = "2020",

doi = "10.1007/978-3-030-45778-5_11",

language = "English",

isbn = "9783030457778",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "154--174",

editor = "Selma Boumerdassi and {\'E}ric Renault and Paul M{\"u}hlethaler",

booktitle = "Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers",

}

Rao, NSV, Imam, N, Liu, Z, Kettimuthu, R & Foster, I 2020, Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses. in S Boumerdassi, É Renault & P Mühlethaler (eds), Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12081 LNCS, Springer, pp. 154-174, 2nd International Conference on Machine Learning for Networking, MLN 2019, Paris, France, 12/3/19. https://doi.org/10.1007/978-3-030-45778-5_11

Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses. / Rao, Nageswara S.V.; Imam, Neena; Liu, Zhengchun et al.
Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers. ed. / Selma Boumerdassi; Éric Renault; Paul Mühlethaler. Springer, 2020. p. 154-174 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12081 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses

AU - Rao, Nageswara S.V.

AU - Imam, Neena

AU - Liu, Zhengchun

AU - Kettimuthu, Rajkumar

AU - Foster, Ian

PY - 2020

Y1 - 2020

N2 - Scientific computations are expected to be increasingly distributed across wide-area networks, and Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. Application-level measurements of MPI operations reflect the connection Round-Trip Time (RTT) and loss rate, and machine learning methods have been previously developed to estimate them under deterministic periodic losses. In this paper, we consider more complex, random losses with uniform, Poisson and Gaussian distributions. We study five disparate machine leaning methods, with linear and non-linear, and smooth and non-smooth properties, to estimate RTT and loss rate over 10 Gbps connections with 0–366 ms RTT. The diversity and complexity of these estimators combined with the randomness of losses and TCP’s non-linear response together rule out the selection of a single best among them; instead, we fuse them to retain their design diversity. Overall, the results show that accurate estimates can be generated at low loss rates but become inaccurate at loss rates 10% and higher, thereby illustrating both their strengths and limitations.

AB - Scientific computations are expected to be increasingly distributed across wide-area networks, and Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. Application-level measurements of MPI operations reflect the connection Round-Trip Time (RTT) and loss rate, and machine learning methods have been previously developed to estimate them under deterministic periodic losses. In this paper, we consider more complex, random losses with uniform, Poisson and Gaussian distributions. We study five disparate machine leaning methods, with linear and non-linear, and smooth and non-smooth properties, to estimate RTT and loss rate over 10 Gbps connections with 0–366 ms RTT. The diversity and complexity of these estimators combined with the randomness of losses and TCP’s non-linear response together rule out the selection of a single best among them; instead, we fuse them to retain their design diversity. Overall, the results show that accurate estimates can be generated at low loss rates but become inaccurate at loss rates 10% and higher, thereby illustrating both their strengths and limitations.

KW - Generalization bounds

KW - Information fusion

KW - Loss rate

KW - Machine Learning

KW - Message Passing Interface

KW - Regression

KW - Round Trip Time

UR - http://www.scopus.com/inward/record.url?scp=85084172971&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-45778-5_11

DO - 10.1007/978-3-030-45778-5_11

M3 - Conference contribution

AN - SCOPUS:85084172971

SN - 9783030457778

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 154

EP - 174

BT - Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers

A2 - Boumerdassi, Selma

A2 - Renault, Éric

A2 - Mühlethaler, Paul

PB - Springer

T2 - 2nd International Conference on Machine Learning for Networking, MLN 2019

Y2 - 3 December 2019 through 5 December 2019

ER -

Rao NSV, Imam N, Liu Z, Kettimuthu R, Foster I. Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses. In Boumerdassi S, Renault É, Mühlethaler P, editors, Machine Learning for Networking - 2nd IFIP TC 6 International Conference, MLN 2019, Revised Selected Papers. Springer. 2020. p. 154-174. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-45778-5_11

Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this