Profiling optimization for big data transfer over dedicated channels

Daqing Yun; Chase Q. Wu; Nageswara S.V. Rao; Qiang Liu; Rajkumar Kettimuthu; Eun Sung Jung

doi:10.1109/ICCCN.2016.7568562

Profiling optimization for big data transfer over dedicated channels

Daqing Yun, Chase Q. Wu, Nageswara S.V. Rao, Qiang Liu, Rajkumar Kettimuthu, Eun Sung Jung

Autonomous and Complex Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

The transfer of big data is increasingly supported by dedicated channels in high-performance networks, where transport protocols play an important role in maximizing application-level throughput and link utilization. The performance of transport protocols largely depend on their control parameter settings, but it is prohibitively time consuming to conduct an exhaustive search in a large parameter space to find the best set of parameter values. We propose FastProf, a stochastic approximation-based transport profiler, to quickly determine the optimal operational zone of a given data transfer protocol/method over dedicated channels. We implement and test the proposed method using both emulations based on real-life performance measurements and experiments over physical connections with short (2ms) and long (380ms) delays. Both the emulation and experimental results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of end-to-end throughput performance with the exhaustive search-based approach.

Original language	English
Title of host publication	2016 25th International Conference on Computer Communications and Networks, ICCCN 2016
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781509022793
DOIs	https://doi.org/10.1109/ICCCN.2016.7568562
State	Published - Sep 14 2016
Event	25th International Conference on Computer Communications and Networks, ICCCN 2016 - Waikoloa, United States Duration: Aug 1 2016 → Aug 4 2016

Publication series

Name	2016 25th International Conference on Computer Communications and Networks, ICCCN 2016

Conference

Conference	25th International Conference on Computer Communications and Networks, ICCCN 2016
Country/Territory	United States
City	Waikoloa
Period	08/1/16 → 08/4/16

Funding

ACKNOWLEDGMENT: This research is sponsored by U.S. National Science Foundation under Grant No. CNS-1560698 and Oak Ridge National Laboratory, U.S. Department of Energy, under Contract No. DE-AC05-00OR22725/4000141963 with New Jersey Institute of Technology. This work was completed in part with resources provided by the University of Chicago Research Computing Center.

Funders	Funder number
New Jersey Institute of Technology
U.S. National Science Foundation	CNS-1560698
University of Chicago Research Computing Center
U.S. Department of Energy	DE-AC05-00OR22725/4000141963
Oak Ridge National Laboratory

Keywords

Big data transfer
Dedicated channels
High-performance networks
Profiling
Stochastic approximation

Access to Document

10.1109/ICCCN.2016.7568562

Cite this

Yun, D., Wu, C. Q., Rao, N. S. V., Liu, Q., Kettimuthu, R., & Jung, E. S. (2016). Profiling optimization for big data transfer over dedicated channels. In 2016 25th International Conference on Computer Communications and Networks, ICCCN 2016 Article 7568562 (2016 25th International Conference on Computer Communications and Networks, ICCCN 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCCN.2016.7568562

@inproceedings{9de4e8e76f1143f7b4579311bf6b0d8b,

title = "Profiling optimization for big data transfer over dedicated channels",

abstract = "The transfer of big data is increasingly supported by dedicated channels in high-performance networks, where transport protocols play an important role in maximizing application-level throughput and link utilization. The performance of transport protocols largely depend on their control parameter settings, but it is prohibitively time consuming to conduct an exhaustive search in a large parameter space to find the best set of parameter values. We propose FastProf, a stochastic approximation-based transport profiler, to quickly determine the optimal operational zone of a given data transfer protocol/method over dedicated channels. We implement and test the proposed method using both emulations based on real-life performance measurements and experiments over physical connections with short (2ms) and long (380ms) delays. Both the emulation and experimental results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of end-to-end throughput performance with the exhaustive search-based approach.",

keywords = "Big data transfer, Dedicated channels, High-performance networks, Profiling, Stochastic approximation",

author = "Daqing Yun and Wu, {Chase Q.} and Rao, {Nageswara S.V.} and Qiang Liu and Rajkumar Kettimuthu and Jung, {Eun Sung}",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 25th International Conference on Computer Communications and Networks, ICCCN 2016 ; Conference date: 01-08-2016 Through 04-08-2016",

year = "2016",

month = sep,

day = "14",

doi = "10.1109/ICCCN.2016.7568562",

language = "English",

series = "2016 25th International Conference on Computer Communications and Networks, ICCCN 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2016 25th International Conference on Computer Communications and Networks, ICCCN 2016",

}

Yun, D, Wu, CQ, Rao, NSV, Liu, Q, Kettimuthu, R & Jung, ES 2016, Profiling optimization for big data transfer over dedicated channels. in 2016 25th International Conference on Computer Communications and Networks, ICCCN 2016., 7568562, 2016 25th International Conference on Computer Communications and Networks, ICCCN 2016, Institute of Electrical and Electronics Engineers Inc., 25th International Conference on Computer Communications and Networks, ICCCN 2016, Waikoloa, United States, 08/1/16. https://doi.org/10.1109/ICCCN.2016.7568562

Profiling optimization for big data transfer over dedicated channels. / Yun, Daqing; Wu, Chase Q.; Rao, Nageswara S.V. et al.
2016 25th International Conference on Computer Communications and Networks, ICCCN 2016. Institute of Electrical and Electronics Engineers Inc., 2016. 7568562 (2016 25th International Conference on Computer Communications and Networks, ICCCN 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Profiling optimization for big data transfer over dedicated channels

AU - Yun, Daqing

AU - Wu, Chase Q.

AU - Rao, Nageswara S.V.

AU - Liu, Qiang

AU - Kettimuthu, Rajkumar

AU - Jung, Eun Sung

PY - 2016/9/14

Y1 - 2016/9/14

N2 - The transfer of big data is increasingly supported by dedicated channels in high-performance networks, where transport protocols play an important role in maximizing application-level throughput and link utilization. The performance of transport protocols largely depend on their control parameter settings, but it is prohibitively time consuming to conduct an exhaustive search in a large parameter space to find the best set of parameter values. We propose FastProf, a stochastic approximation-based transport profiler, to quickly determine the optimal operational zone of a given data transfer protocol/method over dedicated channels. We implement and test the proposed method using both emulations based on real-life performance measurements and experiments over physical connections with short (2ms) and long (380ms) delays. Both the emulation and experimental results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of end-to-end throughput performance with the exhaustive search-based approach.

AB - The transfer of big data is increasingly supported by dedicated channels in high-performance networks, where transport protocols play an important role in maximizing application-level throughput and link utilization. The performance of transport protocols largely depend on their control parameter settings, but it is prohibitively time consuming to conduct an exhaustive search in a large parameter space to find the best set of parameter values. We propose FastProf, a stochastic approximation-based transport profiler, to quickly determine the optimal operational zone of a given data transfer protocol/method over dedicated channels. We implement and test the proposed method using both emulations based on real-life performance measurements and experiments over physical connections with short (2ms) and long (380ms) delays. Both the emulation and experimental results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of end-to-end throughput performance with the exhaustive search-based approach.

KW - Big data transfer

KW - Dedicated channels

KW - High-performance networks

KW - Profiling

KW - Stochastic approximation

UR - http://www.scopus.com/inward/record.url?scp=84991810786&partnerID=8YFLogxK

U2 - 10.1109/ICCCN.2016.7568562

DO - 10.1109/ICCCN.2016.7568562

M3 - Conference contribution

AN - SCOPUS:84991810786

T3 - 2016 25th International Conference on Computer Communications and Networks, ICCCN 2016

BT - 2016 25th International Conference on Computer Communications and Networks, ICCCN 2016

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 25th International Conference on Computer Communications and Networks, ICCCN 2016

Y2 - 1 August 2016 through 4 August 2016

ER -

Yun D, Wu CQ, Rao NSV, Liu Q, Kettimuthu R, Jung ES. Profiling optimization for big data transfer over dedicated channels. In 2016 25th International Conference on Computer Communications and Networks, ICCCN 2016. Institute of Electrical and Electronics Engineers Inc. 2016. 7568562. (2016 25th International Conference on Computer Communications and Networks, ICCCN 2016). doi: 10.1109/ICCCN.2016.7568562

Profiling optimization for big data transfer over dedicated channels

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this