Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems

Anchen Chai; Sorina Camarasu-Pop; Tristan Glatard; Hugues Benoit-Cattin; Frédéric Suter

doi:10.1007/978-3-030-10549-5_32

Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems

Anchen Chai, Sorina Camarasu-Pop, Tristan Glatard, Hugues Benoit-Cattin, Frédéric Suter

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

File replication is widely used to reduce file transfer times and improve data availability in large distributed systems. Replication techniques are often evaluated through simulations, however, most simulation platform models are oversimplified, which questions the applicability of the findings to real systems. In this paper, we investigate how platform models influence the performance of file replication strategies on large heterogeneous distributed systems, based on common existing techniques such as prestaging and dynamic replication. The novelty of our study resides in our evaluation using a realistic simulator. We consider two platform models: a simple hierarchical model and a detailed model built from execution traces. Our results show that conclusions depend on the modeling of the platform and its capacity to capture the characteristics of the targeted production infrastructure. We also derive recommendations for the implementation of an optimized data management strategy in a scientific gateway for medical image analysis.

Original language	English
Title of host publication	Euro-Par 2018
Subtitle of host publication	Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers
Editors	Gabriele Mencagli, Dora B. Heras
Publisher	Springer Verlag
Pages	409-420
Number of pages	12
ISBN (Print)	9783030105488
DOIs	https://doi.org/10.1007/978-3-030-10549-5_32
State	Published - 2019
Externally published	Yes
Event	24th International Conference on Parallel and Distributed Computing, Euro-Par 2018 - Turin, Italy Duration: Aug 27 2018 → Aug 28 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11339 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Parallel and Distributed Computing, Euro-Par 2018
Country/Territory	Italy
City	Turin
Period	08/27/18 → 08/28/18

Funding

Acknowledgements. This work is partially supported by the LABEX PRIMES This work is partially supported by the LABEX PRIMES (ANR-11-LABX-0063) of Université de Lyon, within the program “Investissements d’Avenir” (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR). The authors also thank EGI and France Grilles for their support and the provided resources.

Keywords

Evaluation
File replication
Platform model
Realistic simulation

Access to Document

10.1007/978-3-030-10549-5_32

Cite this

Chai, A., Camarasu-Pop, S., Glatard, T., Benoit-Cattin, H., & Suter, F. (2019). Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems. In G. Mencagli, & D. B. Heras (Eds.), Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers (pp. 409-420). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11339 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-10549-5_32

Chai, Anchen ; Camarasu-Pop, Sorina ; Glatard, Tristan et al. / Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems. Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. editor / Gabriele Mencagli ; Dora B. Heras. Springer Verlag, 2019. pp. 409-420 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a555902dc20840b38e2c265373969bd5,

title = "Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems",

abstract = "File replication is widely used to reduce file transfer times and improve data availability in large distributed systems. Replication techniques are often evaluated through simulations, however, most simulation platform models are oversimplified, which questions the applicability of the findings to real systems. In this paper, we investigate how platform models influence the performance of file replication strategies on large heterogeneous distributed systems, based on common existing techniques such as prestaging and dynamic replication. The novelty of our study resides in our evaluation using a realistic simulator. We consider two platform models: a simple hierarchical model and a detailed model built from execution traces. Our results show that conclusions depend on the modeling of the platform and its capacity to capture the characteristics of the targeted production infrastructure. We also derive recommendations for the implementation of an optimized data management strategy in a scientific gateway for medical image analysis.",

keywords = "Evaluation, File replication, Platform model, Realistic simulation",

author = "Anchen Chai and Sorina Camarasu-Pop and Tristan Glatard and Hugues Benoit-Cattin and Fr{\'e}d{\'e}ric Suter",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 24th International Conference on Parallel and Distributed Computing, Euro-Par 2018 ; Conference date: 27-08-2018 Through 28-08-2018",

year = "2019",

doi = "10.1007/978-3-030-10549-5\_32",

language = "English",

isbn = "9783030105488",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "409--420",

editor = "Gabriele Mencagli and Heras, \{Dora B.\}",

booktitle = "Euro-Par 2018",

}

Chai, A, Camarasu-Pop, S, Glatard, T, Benoit-Cattin, H & Suter, F 2019, Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems. in G Mencagli & DB Heras (eds), Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11339 LNCS, Springer Verlag, pp. 409-420, 24th International Conference on Parallel and Distributed Computing, Euro-Par 2018, Turin, Italy, 08/27/18. https://doi.org/10.1007/978-3-030-10549-5_32

Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems. / Chai, Anchen; Camarasu-Pop, Sorina; Glatard, Tristan et al.
Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. ed. / Gabriele Mencagli; Dora B. Heras. Springer Verlag, 2019. p. 409-420 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11339 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems

AU - Chai, Anchen

AU - Camarasu-Pop, Sorina

AU - Glatard, Tristan

AU - Benoit-Cattin, Hugues

AU - Suter, Frédéric

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - File replication is widely used to reduce file transfer times and improve data availability in large distributed systems. Replication techniques are often evaluated through simulations, however, most simulation platform models are oversimplified, which questions the applicability of the findings to real systems. In this paper, we investigate how platform models influence the performance of file replication strategies on large heterogeneous distributed systems, based on common existing techniques such as prestaging and dynamic replication. The novelty of our study resides in our evaluation using a realistic simulator. We consider two platform models: a simple hierarchical model and a detailed model built from execution traces. Our results show that conclusions depend on the modeling of the platform and its capacity to capture the characteristics of the targeted production infrastructure. We also derive recommendations for the implementation of an optimized data management strategy in a scientific gateway for medical image analysis.

AB - File replication is widely used to reduce file transfer times and improve data availability in large distributed systems. Replication techniques are often evaluated through simulations, however, most simulation platform models are oversimplified, which questions the applicability of the findings to real systems. In this paper, we investigate how platform models influence the performance of file replication strategies on large heterogeneous distributed systems, based on common existing techniques such as prestaging and dynamic replication. The novelty of our study resides in our evaluation using a realistic simulator. We consider two platform models: a simple hierarchical model and a detailed model built from execution traces. Our results show that conclusions depend on the modeling of the platform and its capacity to capture the characteristics of the targeted production infrastructure. We also derive recommendations for the implementation of an optimized data management strategy in a scientific gateway for medical image analysis.

KW - Evaluation

KW - File replication

KW - Platform model

KW - Realistic simulation

UR - https://www.scopus.com/pages/publications/85061759885

U2 - 10.1007/978-3-030-10549-5_32

DO - 10.1007/978-3-030-10549-5_32

M3 - Conference contribution

AN - SCOPUS:85061759885

SN - 9783030105488

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 409

EP - 420

BT - Euro-Par 2018

A2 - Mencagli, Gabriele

A2 - Heras, Dora B.

PB - Springer Verlag

T2 - 24th International Conference on Parallel and Distributed Computing, Euro-Par 2018

Y2 - 27 August 2018 through 28 August 2018

ER -

Chai A, Camarasu-Pop S, Glatard T, Benoit-Cattin H, Suter F. Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems. In Mencagli G, Heras DB, editors, Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. Springer Verlag. 2019. p. 409-420. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-10549-5_32

Evaluation through realistic simulations of file replication strategies for large heterogeneous distributed systems

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this