Data Readiness for Scientific AI at Scale

Wesley Brewer; Patrick Widener; Valentine Anantharaj; Feiyi Wang; Tom Beck; Arjun Shankar; Sarp Oral

doi:10.1145/3750720.3757282

Data Readiness for Scientific AI at Scale

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains - climate, nuclear fusion, bio/health, and materials - to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework composed of Data Readiness Levels (raw to AI-ready) and Data Processing Stages (ingest to shard), both tailored to high performance computing (HPC) environments. This framework outlines key challenges in transforming scientific data for scalable AI training, emphasizing transformer-based generative models. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

Original language	English
Title of host publication	54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings
Publisher	Association for Computing Machinery, Inc
Pages	18-24
Number of pages	7
ISBN (Electronic)	9798400721090
DOIs	https://doi.org/10.1145/3750720.3757282
State	Published - Dec 20 2025
Event	54th International Conference on Parallel Processing Workshop, ICPP 2025 - San Diego, United States Duration: Sep 8 2025 → Sep 11 2025

Publication series

Name	54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings

Conference

Conference	54th International Conference on Parallel Processing Workshop, ICPP 2025
Country/Territory	United States
City	San Diego
Period	09/8/25 → 09/11/25

Funding

This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported under the Advanced Scientific Computing Research programs in the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. We would also like to thank Max Lupo Pasini, Jens Glaser, Yashika Ghai, Fernanda Foertter, John Gounley, and Heidi Hanson for helpful information regarding AI readiness challenges for specific domains. Finally, OpenAI's ChatGPT was used to provide editing suggestions for several sentences in the paper, as well as to help analyze some of the preprocessing patterns, given large chunks of code.

Keywords

AI-readiness
bioinformatics
climate science
data preprocessing
fusion research
high-performance computing
materials science
scientific datasets

Access to Document

10.1145/3750720.3757282

Cite this

Brewer, W., Widener, P., Anantharaj, V., Wang, F., Beck, T., Shankar, A., & Oral, S. (2025). Data Readiness for Scientific AI at Scale. In 54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings (pp. 18-24). (54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings). Association for Computing Machinery, Inc. https://doi.org/10.1145/3750720.3757282

@inproceedings{fd45048252cb4ef2878baf7378578ddf,

title = "Data Readiness for Scientific AI at Scale",

abstract = "This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains - climate, nuclear fusion, bio/health, and materials - to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework composed of Data Readiness Levels (raw to AI-ready) and Data Processing Stages (ingest to shard), both tailored to high performance computing (HPC) environments. This framework outlines key challenges in transforming scientific data for scalable AI training, emphasizing transformer-based generative models. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.",

keywords = "AI-readiness, bioinformatics, climate science, data preprocessing, fusion research, high-performance computing, materials science, scientific datasets",

author = "Wesley Brewer and Patrick Widener and Valentine Anantharaj and Feiyi Wang and Tom Beck and Arjun Shankar and Sarp Oral",

note = "Publisher Copyright: {\textcopyright} 2025 Copyright held by the owner/author(s).; 54th International Conference on Parallel Processing Workshop, ICPP 2025 ; Conference date: 08-09-2025 Through 11-09-2025",

year = "2025",

month = dec,

day = "20",

doi = "10.1145/3750720.3757282",

language = "English",

series = "54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings",

publisher = "Association for Computing Machinery, Inc",

pages = "18--24",

booktitle = "54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings",

}

Brewer, W , Widener, P , Anantharaj, V , Wang, F , Beck, T , Shankar, A & Oral, S 2025, Data Readiness for Scientific AI at Scale. in 54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings. 54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings, Association for Computing Machinery, Inc, pp. 18-24, 54th International Conference on Parallel Processing Workshop, ICPP 2025, San Diego, United States, 09/8/25. https://doi.org/10.1145/3750720.3757282

Data Readiness for Scientific AI at Scale. / Brewer, Wesley ; Widener, Patrick ; Anantharaj, Valentine et al.
54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings. Association for Computing Machinery, Inc, 2025. p. 18-24 (54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Data Readiness for Scientific AI at Scale

AU - Brewer, Wesley

AU - Widener, Patrick

AU - Anantharaj, Valentine

AU - Wang, Feiyi

AU - Beck, Tom

AU - Shankar, Arjun

AU - Oral, Sarp

PY - 2025/12/20

Y1 - 2025/12/20

N2 - This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains - climate, nuclear fusion, bio/health, and materials - to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework composed of Data Readiness Levels (raw to AI-ready) and Data Processing Stages (ingest to shard), both tailored to high performance computing (HPC) environments. This framework outlines key challenges in transforming scientific data for scalable AI training, emphasizing transformer-based generative models. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

AB - This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains - climate, nuclear fusion, bio/health, and materials - to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework composed of Data Readiness Levels (raw to AI-ready) and Data Processing Stages (ingest to shard), both tailored to high performance computing (HPC) environments. This framework outlines key challenges in transforming scientific data for scalable AI training, emphasizing transformer-based generative models. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

KW - AI-readiness

KW - bioinformatics

KW - climate science

KW - data preprocessing

KW - fusion research

KW - high-performance computing

KW - materials science

KW - scientific datasets

UR - https://www.scopus.com/pages/publications/105026445344

U2 - 10.1145/3750720.3757282

DO - 10.1145/3750720.3757282

M3 - Conference contribution

AN - SCOPUS:105026445344

T3 - 54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings

SP - 18

EP - 24

BT - 54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings

PB - Association for Computing Machinery, Inc

T2 - 54th International Conference on Parallel Processing Workshop, ICPP 2025

Y2 - 8 September 2025 through 11 September 2025

ER -

Brewer W , Widener P , Anantharaj V , Wang F , Beck T , Shankar A et al. Data Readiness for Scientific AI at Scale. In 54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings. Association for Computing Machinery, Inc. 2025. p. 18-24. (54th International Conference on Parallel Processing, ICPP 2025 - Workshops Proceedings). doi: 10.1145/3750720.3757282

Data Readiness for Scientific AI at Scale

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this