Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing

Shengyu Feng; Jaehyung Kim; Yiming Yang; Joseph Boudreau; Tasnuva Chowdhury; Adolfy Hoisie; Raees Khan; Ozgur O. Kilic; Scott Klasky; Tatiana Korchuganova; Paul Nilsson; Verena Ingrid Martinez Outschoorn; David K. Park; Norbert Podhorszki; Yihui Ren; Frédéric Suter; Sairam Sri Vatsavai; Wei Yang; Shinjae Yoo; Tadashi Maeno; Alexei Klimentov

doi:10.1016/j.future.2025.108075

Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing

Shengyu Feng, Jaehyung Kim, Yiming Yang, Joseph Boudreau, Tasnuva Chowdhury, Adolfy Hoisie, Raees Khan, Ozgur O. Kilic, Scott Klasky, Tatiana Korchuganova, Paul Nilsson, Verena Ingrid Martinez Outschoorn, David K. Park, Norbert Podhorszki, Yihui Ren, Frédéric Suter, Sairam Sri Vatsavai, Wei Yang, Shinjae Yoo, Tadashi MaenoAlexei Klimentov

Workflow Systems

Research output: Contribution to journal › Article › peer-review

Abstract

This paper presents a novel approach to the joint optimization of job scheduling and data allocation in grid computing environments. We formulate this joint optimization problem as a mixed integer quadratically constrained program. To tackle the nonlinearity in the constraint, we alternatively fix a subset of decision variables and optimize the remaining ones via Mixed Integer Linear Programming (MILP). We solve the MILP problem at each iteration via an off-the-shelf MILP solver. Our experimental results show that our method significantly outperforms existing heuristic methods, employing either independent optimization or joint optimization strategies. We have also verified the generalization ability of our method over grid environments with various sizes and its high robustness to the algorithm setting.

Original language	English
Article number	108075
Journal	Future Generation Computer Systems
Volume	175
DOIs	https://doi.org/10.1016/j.future.2025.108075
State	Published - Feb 2026

Funding

This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research under Award Number DE-SC-0012704. This work was done in collaboration with the distributed computing research and development program within the ATLAS Collaboration. We thank our ATLAS colleagues, and in particular, the contributions of the ATLAS Distributed Computing team for their support. We would also like to express our deepest gratitude to Prof. Kaushik De at University of Texas at Arlington.

Keywords

Date allocation
Grid computing environments
High performance computing
Job scheduling
Mixed integer linear programming

Access to Document

10.1016/j.future.2025.108075

Cite this

Feng, S., Kim, J., Yang, Y., Boudreau, J., Chowdhury, T., Hoisie, A., Khan, R., Kilic, O. O., Klasky, S., Korchuganova, T., Nilsson, P., Martinez Outschoorn, V. I., Park, D. K., Podhorszki, N., Ren, Y., Suter, F., Sri Vatsavai, S., Yang, W., Yoo, S., ... Klimentov, A. (2026). Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing. Future Generation Computer Systems, 175, Article 108075. https://doi.org/10.1016/j.future.2025.108075

@article{c781eedb9972458fa912074bf7e0ae20,

title = "Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing",

abstract = "This paper presents a novel approach to the joint optimization of job scheduling and data allocation in grid computing environments. We formulate this joint optimization problem as a mixed integer quadratically constrained program. To tackle the nonlinearity in the constraint, we alternatively fix a subset of decision variables and optimize the remaining ones via Mixed Integer Linear Programming (MILP). We solve the MILP problem at each iteration via an off-the-shelf MILP solver. Our experimental results show that our method significantly outperforms existing heuristic methods, employing either independent optimization or joint optimization strategies. We have also verified the generalization ability of our method over grid environments with various sizes and its high robustness to the algorithm setting.",

keywords = "Date allocation, Grid computing environments, High performance computing, Job scheduling, Mixed integer linear programming",

author = "Shengyu Feng and Jaehyung Kim and Yiming Yang and Joseph Boudreau and Tasnuva Chowdhury and Adolfy Hoisie and Raees Khan and Kilic, \{Ozgur O.\} and Scott Klasky and Tatiana Korchuganova and Paul Nilsson and \{Martinez Outschoorn\}, \{Verena Ingrid\} and Park, \{David K.\} and Norbert Podhorszki and Yihui Ren and Fr{\'e}d{\'e}ric Suter and \{Sri Vatsavai\}, Sairam and Wei Yang and Shinjae Yoo and Tadashi Maeno and Alexei Klimentov",

note = "Publisher Copyright: {\textcopyright} 2025 The Author(s)",

year = "2026",

month = feb,

doi = "10.1016/j.future.2025.108075",

language = "English",

volume = "175",

journal = "Future Generation Computer Systems",

issn = "0167-739X",

publisher = "Elsevier",

}

Feng, S, Kim, J, Yang, Y, Boudreau, J, Chowdhury, T, Hoisie, A, Khan, R, Kilic, OO, Klasky, S, Korchuganova, T, Nilsson, P, Martinez Outschoorn, VI, Park, DK, Podhorszki, N, Ren, Y, Suter, F, Sri Vatsavai, S, Yang, W, Yoo, S, Maeno, T & Klimentov, A 2026, 'Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing', Future Generation Computer Systems, vol. 175, 108075. https://doi.org/10.1016/j.future.2025.108075

TY - JOUR

T1 - Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing

AU - Feng, Shengyu

AU - Kim, Jaehyung

AU - Yang, Yiming

AU - Boudreau, Joseph

AU - Chowdhury, Tasnuva

AU - Hoisie, Adolfy

AU - Khan, Raees

AU - Kilic, Ozgur O.

AU - Klasky, Scott

AU - Korchuganova, Tatiana

AU - Nilsson, Paul

AU - Martinez Outschoorn, Verena Ingrid

AU - Park, David K.

AU - Podhorszki, Norbert

AU - Ren, Yihui

AU - Suter, Frédéric

AU - Sri Vatsavai, Sairam

AU - Yang, Wei

AU - Yoo, Shinjae

AU - Maeno, Tadashi

AU - Klimentov, Alexei

PY - 2026/2

Y1 - 2026/2

N2 - This paper presents a novel approach to the joint optimization of job scheduling and data allocation in grid computing environments. We formulate this joint optimization problem as a mixed integer quadratically constrained program. To tackle the nonlinearity in the constraint, we alternatively fix a subset of decision variables and optimize the remaining ones via Mixed Integer Linear Programming (MILP). We solve the MILP problem at each iteration via an off-the-shelf MILP solver. Our experimental results show that our method significantly outperforms existing heuristic methods, employing either independent optimization or joint optimization strategies. We have also verified the generalization ability of our method over grid environments with various sizes and its high robustness to the algorithm setting.

AB - This paper presents a novel approach to the joint optimization of job scheduling and data allocation in grid computing environments. We formulate this joint optimization problem as a mixed integer quadratically constrained program. To tackle the nonlinearity in the constraint, we alternatively fix a subset of decision variables and optimize the remaining ones via Mixed Integer Linear Programming (MILP). We solve the MILP problem at each iteration via an off-the-shelf MILP solver. Our experimental results show that our method significantly outperforms existing heuristic methods, employing either independent optimization or joint optimization strategies. We have also verified the generalization ability of our method over grid environments with various sizes and its high robustness to the algorithm setting.

KW - Date allocation

KW - Grid computing environments

KW - High performance computing

KW - Job scheduling

KW - Mixed integer linear programming

UR - https://www.scopus.com/pages/publications/105014528071

U2 - 10.1016/j.future.2025.108075

DO - 10.1016/j.future.2025.108075

M3 - Article

AN - SCOPUS:105014528071

SN - 0167-739X

VL - 175

JO - Future Generation Computer Systems

JF - Future Generation Computer Systems

M1 - 108075

ER -

Alternative mixed integer linear programming optimization for joint job scheduling and data allocation in grid computing

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this