Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations

Yuanlai Liu; Zhengchun Liu; Rajkumar Kettimuthu; Nageswara Rao; Zizhong Chen; Ian Foster

doi:10.1109/CCGRID.2019.00023

Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations

Yuanlai Liu, Zhengchun Liu, Rajkumar Kettimuthu, Nageswara Rao, Zizhong Chen, Ian Foster

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

23 Scopus citations

Abstract

Wide area file transfers play an important role in many science applications. File transfer tools typically deliver the highest performance for datasets with a small number of large files, but many science datasets consist of many small files. Thus it is important to understand the factors that contribute to the decrease in wide area data transfer performance for datasets with many small files. To this end, we (i) benchmark the performance of subsystems involved in end-to-end file transfer between two HPC facilities for a many-file dataset that is representative of production science transfers; (ii) characterize the per-file overhead introduced by different subsystems; (iii) identify potential dependencies and bottlenecks; (iv) study the effectiveness of transferring many files concurrently as a means of reducing per-file overheads; and (v) prototype a prefetching mechanism as an alternative of concurrency to reduce the per-file overhead on source storage system. We show that both concurrency and prefetching can help reduce the per-file overhead significantly. A reasonable level of concurrency combined with prefetching can bring the per-file overhead down to a negligible level.

Original language	English
Title of host publication	Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	122-131
Number of pages	10
ISBN (Electronic)	9781728109121
DOIs	https://doi.org/10.1109/CCGRID.2019.00023
State	Published - May 2019
Event	19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019 - Larnaca, Cyprus Duration: May 14 2019 → May 17 2019

Publication series

Name	Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019

Conference

Conference	19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019
Country/Territory	Cyprus
City	Larnaca
Period	05/14/19 → 05/17/19

Funding

This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract number DE-AC02-06CH11357. Z. Liu and Y. Liu contributed equally to this research. We gratefully acknowledge the National Energy Research Scientific Computing Center and Argonne Leadership Computing Facility for providing us resources.

Keywords

Data transfer
GridFTP
Model
Optimization

Access to Document

10.1109/CCGRID.2019.00023

Cite this

Liu, Y., Liu, Z., Kettimuthu, R., Rao, N., Chen, Z., & Foster, I. (2019). Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations. In Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019 (pp. 122-131). Article 8752877 (Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCGRID.2019.00023

Liu, Yuanlai ; Liu, Zhengchun ; Kettimuthu, Rajkumar et al. / Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations. Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 122-131 (Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019).

@inproceedings{5bbf5f001599453f894f7d04f6c768d7,

title = "Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations",

abstract = "Wide area file transfers play an important role in many science applications. File transfer tools typically deliver the highest performance for datasets with a small number of large files, but many science datasets consist of many small files. Thus it is important to understand the factors that contribute to the decrease in wide area data transfer performance for datasets with many small files. To this end, we (i) benchmark the performance of subsystems involved in end-to-end file transfer between two HPC facilities for a many-file dataset that is representative of production science transfers; (ii) characterize the per-file overhead introduced by different subsystems; (iii) identify potential dependencies and bottlenecks; (iv) study the effectiveness of transferring many files concurrently as a means of reducing per-file overheads; and (v) prototype a prefetching mechanism as an alternative of concurrency to reduce the per-file overhead on source storage system. We show that both concurrency and prefetching can help reduce the per-file overhead significantly. A reasonable level of concurrency combined with prefetching can bring the per-file overhead down to a negligible level.",

keywords = "Data transfer, GridFTP, Model, Optimization",

author = "Yuanlai Liu and Zhengchun Liu and Rajkumar Kettimuthu and Nageswara Rao and Zizhong Chen and Ian Foster",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019 ; Conference date: 14-05-2019 Through 17-05-2019",

year = "2019",

month = may,

doi = "10.1109/CCGRID.2019.00023",

language = "English",

series = "Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "122--131",

booktitle = "Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019",

}

Liu, Y, Liu, Z, Kettimuthu, R, Rao, N, Chen, Z & Foster, I 2019, Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations. in Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019., 8752877, Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019, Institute of Electrical and Electronics Engineers Inc., pp. 122-131, 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019, Larnaca, Cyprus, 05/14/19. https://doi.org/10.1109/CCGRID.2019.00023

Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations. / Liu, Yuanlai; Liu, Zhengchun; Kettimuthu, Rajkumar et al.
Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 122-131 8752877 (Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations

AU - Liu, Yuanlai

AU - Liu, Zhengchun

AU - Kettimuthu, Rajkumar

AU - Rao, Nageswara

AU - Chen, Zizhong

AU - Foster, Ian

PY - 2019/5

Y1 - 2019/5

N2 - Wide area file transfers play an important role in many science applications. File transfer tools typically deliver the highest performance for datasets with a small number of large files, but many science datasets consist of many small files. Thus it is important to understand the factors that contribute to the decrease in wide area data transfer performance for datasets with many small files. To this end, we (i) benchmark the performance of subsystems involved in end-to-end file transfer between two HPC facilities for a many-file dataset that is representative of production science transfers; (ii) characterize the per-file overhead introduced by different subsystems; (iii) identify potential dependencies and bottlenecks; (iv) study the effectiveness of transferring many files concurrently as a means of reducing per-file overheads; and (v) prototype a prefetching mechanism as an alternative of concurrency to reduce the per-file overhead on source storage system. We show that both concurrency and prefetching can help reduce the per-file overhead significantly. A reasonable level of concurrency combined with prefetching can bring the per-file overhead down to a negligible level.

AB - Wide area file transfers play an important role in many science applications. File transfer tools typically deliver the highest performance for datasets with a small number of large files, but many science datasets consist of many small files. Thus it is important to understand the factors that contribute to the decrease in wide area data transfer performance for datasets with many small files. To this end, we (i) benchmark the performance of subsystems involved in end-to-end file transfer between two HPC facilities for a many-file dataset that is representative of production science transfers; (ii) characterize the per-file overhead introduced by different subsystems; (iii) identify potential dependencies and bottlenecks; (iv) study the effectiveness of transferring many files concurrently as a means of reducing per-file overheads; and (v) prototype a prefetching mechanism as an alternative of concurrency to reduce the per-file overhead on source storage system. We show that both concurrency and prefetching can help reduce the per-file overhead significantly. A reasonable level of concurrency combined with prefetching can bring the per-file overhead down to a negligible level.

KW - Data transfer

KW - GridFTP

KW - Model

KW - Optimization

UR - http://www.scopus.com/inward/record.url?scp=85069454110&partnerID=8YFLogxK

U2 - 10.1109/CCGRID.2019.00023

DO - 10.1109/CCGRID.2019.00023

M3 - Conference contribution

AN - SCOPUS:85069454110

T3 - Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019

SP - 122

EP - 131

BT - Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019

Y2 - 14 May 2019 through 17 May 2019

ER -

Liu Y, Liu Z, Kettimuthu R, Rao N, Chen Z, Foster I. Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations. In Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 122-131. 8752877. (Proceedings - 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2019). doi: 10.1109/CCGRID.2019.00023

Data transfer between scientific facilities - Bottleneck analysis, insights and optimizations

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this