yProv4ML: Effortless provenance tracking for machine learning systems

G. Padovani; V. Anantharaj; S. Fiore

doi:10.1016/j.softx.2025.102298

yProv4ML: Effortless provenance tracking for machine learning systems

G. Padovani
, V. Anantharaj
, S. Fiore

Workflow and Ecosystem Services

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.

Original language	English
Article number	102298
Journal	SoftwareX
Volume	31
DOIs	https://doi.org/10.1016/j.softx.2025.102298
State	Published - Sep 2025

Funding

This work was partially funded under the National Recovery and Resilience Plan (NRRP), Mission 4 Component 2 Investment 1.4 - Call for tender No. 1031 of 17/06/2022 of Italian Ministry for University and Research funded by the European Union – NextGenerationEU (proj. nr. CN_00000013) and the EU InterTwin project (Grant Agreement 101058386). Moreover this research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Keywords

Machine learning
PROV-JSON
Provenance
Provenance graph
yProv4ML

Access to Document

10.1016/j.softx.2025.102298

Cite this

@article{d8728e6664bb4aeda7e93deea85ff9ea,

title = "yProv4ML: Effortless provenance tracking for machine learning systems",

abstract = "The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.",

keywords = "Machine learning, PROV-JSON, Provenance, Provenance graph, yProv4ML",

author = "G. Padovani and V. Anantharaj and S. Fiore",

note = "Publisher Copyright: {\textcopyright} 2025",

year = "2025",

month = sep,

doi = "10.1016/j.softx.2025.102298",

language = "English",

volume = "31",

journal = "SoftwareX",

issn = "2352-7110",

publisher = "Elsevier",

}

TY - JOUR

T1 - yProv4ML

T2 - Effortless provenance tracking for machine learning systems

AU - Padovani, G.

AU - Anantharaj, V.

AU - Fiore, S.

PY - 2025/9

Y1 - 2025/9

N2 - The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.

AB - The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.

KW - Machine learning

KW - PROV-JSON

KW - Provenance

KW - Provenance graph

KW - yProv4ML

UR - https://www.scopus.com/pages/publications/105014932292

U2 - 10.1016/j.softx.2025.102298

DO - 10.1016/j.softx.2025.102298

M3 - Article

AN - SCOPUS:105014932292

SN - 2352-7110

VL - 31

JO - SoftwareX

JF - SoftwareX

M1 - 102298

ER -

yProv4ML: Effortless provenance tracking for machine learning systems

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this