OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems

Weijian Zheng; Dali Wang; Fengguang Song

doi:10.1007/978-3-030-50426-7_33

OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems

Weijian Zheng, Dali Wang, Fengguang Song

Earth Systems Modeling

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new open-source RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.

Original language	English
Title of host publication	Computational Science – ICCS 2020 - 20th International Conference, Proceedings
Editors	Valeria V. Krzhizhanovskaya, Gábor Závodszky, Michael H. Lees, Peter M.A. Sloot, Peter M.A. Sloot, Peter M.A. Sloot, Jack J. Dongarra, Sérgio Brissos, João Teixeira
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	439-452
Number of pages	14
ISBN (Print)	9783030504250
DOIs	https://doi.org/10.1007/978-3-030-50426-7_33
State	Published - 2020
Event	20th International Conference on Computational Science, ICCS 2020 - Amsterdam, Netherlands Duration: Jun 3 2020 → Jun 5 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12141 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th International Conference on Computational Science, ICCS 2020
Country/Territory	Netherlands
City	Amsterdam
Period	06/3/20 → 06/5/20

Funding

This research was funded by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research (Interoperable Design of Extreme-scale Application Software).

Keywords

Distributed GPU computing
Graph optimization problems
Open AI software environment
Reinforcement learning

Access to Document

10.1007/978-3-030-50426-7_33

Cite this

Zheng, W., Wang, D., & Song, F. (2020). OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems. In V. V. Krzhizhanovskaya, G. Závodszky, M. H. Lees, P. M. A. Sloot, P. M. A. Sloot, P. M. A. Sloot, J. J. Dongarra, S. Brissos, & J. Teixeira (Eds.), Computational Science – ICCS 2020 - 20th International Conference, Proceedings (pp. 439-452). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12141 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-50426-7_33

Zheng, Weijian ; Wang, Dali ; Song, Fengguang. / OpenGraphGym : A parallel reinforcement learning framework for graph optimization problems. Computational Science – ICCS 2020 - 20th International Conference, Proceedings. editor / Valeria V. Krzhizhanovskaya ; Gábor Závodszky ; Michael H. Lees ; Peter M.A. Sloot ; Peter M.A. Sloot ; Peter M.A. Sloot ; Jack J. Dongarra ; Sérgio Brissos ; João Teixeira. Springer Science and Business Media Deutschland GmbH, 2020. pp. 439-452 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{31ff6b9ea4244c86beeae0ba784a3f1c,

title = "OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems",

abstract = "This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new open-source RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.",

keywords = "Distributed GPU computing, Graph optimization problems, Open AI software environment, Reinforcement learning",

author = "Weijian Zheng and Dali Wang and Fengguang Song",

note = "Publisher Copyright: {\textcopyright} 2020, This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply.; 20th International Conference on Computational Science, ICCS 2020 ; Conference date: 03-06-2020 Through 05-06-2020",

year = "2020",

doi = "10.1007/978-3-030-50426-7_33",

language = "English",

isbn = "9783030504250",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "439--452",

editor = "Krzhizhanovskaya, {Valeria V.} and G{\'a}bor Z{\'a}vodszky and Lees, {Michael H.} and Sloot, {Peter M.A.} and Sloot, {Peter M.A.} and Sloot, {Peter M.A.} and Dongarra, {Jack J.} and S{\'e}rgio Brissos and Jo{\~a}o Teixeira",

booktitle = "Computational Science – ICCS 2020 - 20th International Conference, Proceedings",

}

Zheng, W, Wang, D & Song, F 2020, OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems. in VV Krzhizhanovskaya, G Závodszky, MH Lees, PMA Sloot, PMA Sloot, PMA Sloot, JJ Dongarra, S Brissos & J Teixeira (eds), Computational Science – ICCS 2020 - 20th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12141 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 439-452, 20th International Conference on Computational Science, ICCS 2020, Amsterdam, Netherlands, 06/3/20. https://doi.org/10.1007/978-3-030-50426-7_33

OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems. / Zheng, Weijian; Wang, Dali; Song, Fengguang.
Computational Science – ICCS 2020 - 20th International Conference, Proceedings. ed. / Valeria V. Krzhizhanovskaya; Gábor Závodszky; Michael H. Lees; Peter M.A. Sloot; Peter M.A. Sloot; Peter M.A. Sloot; Jack J. Dongarra; Sérgio Brissos; João Teixeira. Springer Science and Business Media Deutschland GmbH, 2020. p. 439-452 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12141 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - OpenGraphGym

T2 - 20th International Conference on Computational Science, ICCS 2020

AU - Zheng, Weijian

AU - Wang, Dali

AU - Song, Fengguang

PY - 2020

Y1 - 2020

N2 - This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new open-source RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.

AB - This paper presents an open-source, parallel AI environment (named OpenGraphGym) to facilitate the application of reinforcement learning (RL) algorithms to address combinatorial graph optimization problems. This environment incorporates a basic deep reinforcement learning method, and several graph embeddings to capture graph features, it also allows users to rapidly plug in and test new RL algorithms and graph embeddings for graph optimization problems. This new open-source RL framework is targeted at achieving both high performance and high quality of the computed graph solutions. This RL framework forms the foundation of several ongoing research directions, including 1) benchmark works on different RL algorithms and embedding methods for classic graph problems; 2) advanced parallel strategies for extreme-scale graph computations, as well as 3) performance evaluation on real-world graph solutions.

KW - Distributed GPU computing

KW - Graph optimization problems

KW - Open AI software environment

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85087528999&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-50426-7_33

DO - 10.1007/978-3-030-50426-7_33

M3 - Conference contribution

AN - SCOPUS:85087528999

SN - 9783030504250

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 439

EP - 452

BT - Computational Science – ICCS 2020 - 20th International Conference, Proceedings

A2 - Krzhizhanovskaya, Valeria V.

A2 - Závodszky, Gábor

A2 - Lees, Michael H.

A2 - Sloot, Peter M.A.

A2 - Dongarra, Jack J.

A2 - Brissos, Sérgio

A2 - Teixeira, João

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 3 June 2020 through 5 June 2020

ER -

Zheng W, Wang D, Song F. OpenGraphGym: A parallel reinforcement learning framework for graph optimization problems. In Krzhizhanovskaya VV, Závodszky G, Lees MH, Sloot PMA, Sloot PMA, Sloot PMA, Dongarra JJ, Brissos S, Teixeira J, editors, Computational Science – ICCS 2020 - 20th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 439-452. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-50426-7_33