SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS

Lele Luan; Nesar Ramachandra; Sandipp Krishnan Ravi; Anindya Bhaduri; Piyush Pandita; Prasanna Balaprakash; Mihai Anitescu; Changjie Sun; Liping Wang

doi:10.1115/DETC2023110704

SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS

Lele Luan, Nesar Ramachandra, Sandipp Krishnan Ravi, Anindya Bhaduri, Piyush Pandita, Prasanna Balaprakash, Mihai Anitescu, Changjie Sun, Liping Wang

CCSD Planning, Partnerships, Operations

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Modern computational methods involving highly sophisticated mathematical formulations enable several tasks like modeling complex physical phenomena, predicting key properties, and optimizing design. The higher fidelity in these computer models makes it computationally intensive to query them hundreds of times for optimization. One usually relies on a simplified model, albeit at the cost of losing predictive accuracy and precision. Towards this, data-driven surrogate modeling methods have shown much promise in emulating the behavior of expensive computer models. However, a major bottleneck in such methods is the inability to deal with high input dimensionality and the need for relatively large datasets. In certain cases, the high dimensionality of the input space can be attributed to its image-like characteristics, for example, the stress and displacement fields of continuums. With such problems, the input and output quantity of interest are tensors of high dimensionality. Commonly used surrogate modeling methods for such problems suffer from requirements like many computational evaluations that precludes one from performing other numerical tasks like uncertainty quantification and statistical analysis. This work proposes an end-to-end approach that maps a high-dimensional image-like input to an output of high dimensionality or its key statistics. Our approach uses two main frameworks that perform three steps: a) reduce the input and output from a high-dimensional space to a reduced or low-dimensional space, b) model the input-output relationship in the low-dimensional space, and c) enable the incorporation of domain-specific physical constraints as masks. To reduce input dimensionality, we leverage principal component analysis, coupled with two surrogate modeling methods: a) Bayesian hybrid modeling and b) DeepHyper’s deep neural networks. We demonstrate the approach’s applicability to a linear elastic stress field data problem. We perform numerical studies to study the effect of the two end-to-end workflows and the effect of data size. Key insights and conclusions are provided, which can aid such efforts in surrogate modeling and engineering optimization.

Original language	English
Title of host publication	43rd Computers and Information in Engineering Conference (CIE)
Publisher	American Society of Mechanical Engineers (ASME)
ISBN (Electronic)	9780791887295
DOIs	https://doi.org/10.1115/DETC2023110704
State	Published - 2023
Event	ASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023 - Boston, United States Duration: Aug 20 2023 → Aug 23 2023

Publication series

Name	Proceedings of the ASME Design Engineering Technical Conference
Volume	2

Conference

Conference	ASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023
Country/Territory	United States
City	Boston
Period	08/20/23 → 08/23/23

Funding

This material is based upon work supported by the U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) under the Advanced Manufacturing Office, Award Number DE-AC0206H11357. The views expressed herein do not necessarily represent the views of the U.S. Department of Energy or the United States Government. Work at Argonne National Laboratory was supported by the U.S. Department of Energy, Office of High Energy Physics. Argonne, a U.S. Department of Energy Office of Science Laboratory, is operated by UChicago Argonne LLC under contract no. DEAC02-06CH11357. This manuscript has been authored by UTBattelle, LLC under Contract No. DE-AC05-00OR22725 with the US Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/ doe-public-access-plan). Part of the analysis here is carried out on Swing, a GPU system at the Laboratory Computing Resource Center (LCRC) of Argonne National Laboratory. We would also like to thank Dr. Aymeric Moinet, at General Electric Research, for his insights into the ellipsoidal void problem. This material is based upon work supported by the U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) under the Advanced Manufacturing Office, Award Number DE-AC0206H11357. The views expressed herein do not necessarily represent the views of the U.S. Department of Energy or the United States Government. Work at Argonne National Laboratory was supported by the U.S. Department of Energy, Office of High Energy Physics. Argonne, a U.S. Department of Energy Office of Science Laboratory, is operated by UChicago Argonne LLC under contract no. DE-AC02-06CH11357. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the US Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these re- sults of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/ doe-public-access-plan). Part of the analysis here is carried out on Swing, a GPU system at the Laboratory Computing Resource Center (LCRC) of Argonne National Laboratory. We would also like to thank Dr. Aymeric Moinet, at General Electric Research, for his insights into the ellipsoidal void problem.

Keywords

Bayesian hybrid modeling
Deep neural networks
Dimension reduction
Image-based models
Surrogate modeling

Access to Document

10.1115/DETC2023110704

Cite this

Luan, L., Ramachandra, N., Ravi, S. K., Bhaduri, A., Pandita, P., Balaprakash, P., Anitescu, M., Sun, C., & Wang, L. (2023). SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS. In 43rd Computers and Information in Engineering Conference (CIE) Article v002t02a011 (Proceedings of the ASME Design Engineering Technical Conference; Vol. 2). American Society of Mechanical Engineers (ASME). https://doi.org/10.1115/DETC2023110704

Luan, Lele ; Ramachandra, Nesar ; Ravi, Sandipp Krishnan et al. / SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS. 43rd Computers and Information in Engineering Conference (CIE). American Society of Mechanical Engineers (ASME), 2023. (Proceedings of the ASME Design Engineering Technical Conference).

@inproceedings{381f11f1a0e54ae583a1eb615df0cb22,

title = "SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS",

abstract = "Modern computational methods involving highly sophisticated mathematical formulations enable several tasks like modeling complex physical phenomena, predicting key properties, and optimizing design. The higher fidelity in these computer models makes it computationally intensive to query them hundreds of times for optimization. One usually relies on a simplified model, albeit at the cost of losing predictive accuracy and precision. Towards this, data-driven surrogate modeling methods have shown much promise in emulating the behavior of expensive computer models. However, a major bottleneck in such methods is the inability to deal with high input dimensionality and the need for relatively large datasets. In certain cases, the high dimensionality of the input space can be attributed to its image-like characteristics, for example, the stress and displacement fields of continuums. With such problems, the input and output quantity of interest are tensors of high dimensionality. Commonly used surrogate modeling methods for such problems suffer from requirements like many computational evaluations that precludes one from performing other numerical tasks like uncertainty quantification and statistical analysis. This work proposes an end-to-end approach that maps a high-dimensional image-like input to an output of high dimensionality or its key statistics. Our approach uses two main frameworks that perform three steps: a) reduce the input and output from a high-dimensional space to a reduced or low-dimensional space, b) model the input-output relationship in the low-dimensional space, and c) enable the incorporation of domain-specific physical constraints as masks. To reduce input dimensionality, we leverage principal component analysis, coupled with two surrogate modeling methods: a) Bayesian hybrid modeling and b) DeepHyper{\textquoteright}s deep neural networks. We demonstrate the approach{\textquoteright}s applicability to a linear elastic stress field data problem. We perform numerical studies to study the effect of the two end-to-end workflows and the effect of data size. Key insights and conclusions are provided, which can aid such efforts in surrogate modeling and engineering optimization.",

keywords = "Bayesian hybrid modeling, Deep neural networks, Dimension reduction, Image-based models, Surrogate modeling",

author = "Lele Luan and Nesar Ramachandra and Ravi, {Sandipp Krishnan} and Anindya Bhaduri and Piyush Pandita and Prasanna Balaprakash and Mihai Anitescu and Changjie Sun and Liping Wang",

note = "Publisher Copyright: Copyright {\textcopyright} 2023 by The United States Government.; ASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023 ; Conference date: 20-08-2023 Through 23-08-2023",

year = "2023",

doi = "10.1115/DETC2023110704",

language = "English",

series = "Proceedings of the ASME Design Engineering Technical Conference",

publisher = "American Society of Mechanical Engineers (ASME)",

booktitle = "43rd Computers and Information in Engineering Conference (CIE)",

}

Luan, L, Ramachandra, N, Ravi, SK, Bhaduri, A, Pandita, P, Balaprakash, P, Anitescu, M, Sun, C & Wang, L 2023, SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS. in 43rd Computers and Information in Engineering Conference (CIE)., v002t02a011, Proceedings of the ASME Design Engineering Technical Conference, vol. 2, American Society of Mechanical Engineers (ASME), ASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023, Boston, United States, 08/20/23. https://doi.org/10.1115/DETC2023110704

SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS. / Luan, Lele; Ramachandra, Nesar; Ravi, Sandipp Krishnan et al.
43rd Computers and Information in Engineering Conference (CIE). American Society of Mechanical Engineers (ASME), 2023. v002t02a011 (Proceedings of the ASME Design Engineering Technical Conference; Vol. 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS

AU - Luan, Lele

AU - Ramachandra, Nesar

AU - Ravi, Sandipp Krishnan

AU - Bhaduri, Anindya

AU - Pandita, Piyush

AU - Balaprakash, Prasanna

AU - Anitescu, Mihai

AU - Sun, Changjie

AU - Wang, Liping

PY - 2023

Y1 - 2023

N2 - Modern computational methods involving highly sophisticated mathematical formulations enable several tasks like modeling complex physical phenomena, predicting key properties, and optimizing design. The higher fidelity in these computer models makes it computationally intensive to query them hundreds of times for optimization. One usually relies on a simplified model, albeit at the cost of losing predictive accuracy and precision. Towards this, data-driven surrogate modeling methods have shown much promise in emulating the behavior of expensive computer models. However, a major bottleneck in such methods is the inability to deal with high input dimensionality and the need for relatively large datasets. In certain cases, the high dimensionality of the input space can be attributed to its image-like characteristics, for example, the stress and displacement fields of continuums. With such problems, the input and output quantity of interest are tensors of high dimensionality. Commonly used surrogate modeling methods for such problems suffer from requirements like many computational evaluations that precludes one from performing other numerical tasks like uncertainty quantification and statistical analysis. This work proposes an end-to-end approach that maps a high-dimensional image-like input to an output of high dimensionality or its key statistics. Our approach uses two main frameworks that perform three steps: a) reduce the input and output from a high-dimensional space to a reduced or low-dimensional space, b) model the input-output relationship in the low-dimensional space, and c) enable the incorporation of domain-specific physical constraints as masks. To reduce input dimensionality, we leverage principal component analysis, coupled with two surrogate modeling methods: a) Bayesian hybrid modeling and b) DeepHyper’s deep neural networks. We demonstrate the approach’s applicability to a linear elastic stress field data problem. We perform numerical studies to study the effect of the two end-to-end workflows and the effect of data size. Key insights and conclusions are provided, which can aid such efforts in surrogate modeling and engineering optimization.

AB - Modern computational methods involving highly sophisticated mathematical formulations enable several tasks like modeling complex physical phenomena, predicting key properties, and optimizing design. The higher fidelity in these computer models makes it computationally intensive to query them hundreds of times for optimization. One usually relies on a simplified model, albeit at the cost of losing predictive accuracy and precision. Towards this, data-driven surrogate modeling methods have shown much promise in emulating the behavior of expensive computer models. However, a major bottleneck in such methods is the inability to deal with high input dimensionality and the need for relatively large datasets. In certain cases, the high dimensionality of the input space can be attributed to its image-like characteristics, for example, the stress and displacement fields of continuums. With such problems, the input and output quantity of interest are tensors of high dimensionality. Commonly used surrogate modeling methods for such problems suffer from requirements like many computational evaluations that precludes one from performing other numerical tasks like uncertainty quantification and statistical analysis. This work proposes an end-to-end approach that maps a high-dimensional image-like input to an output of high dimensionality or its key statistics. Our approach uses two main frameworks that perform three steps: a) reduce the input and output from a high-dimensional space to a reduced or low-dimensional space, b) model the input-output relationship in the low-dimensional space, and c) enable the incorporation of domain-specific physical constraints as masks. To reduce input dimensionality, we leverage principal component analysis, coupled with two surrogate modeling methods: a) Bayesian hybrid modeling and b) DeepHyper’s deep neural networks. We demonstrate the approach’s applicability to a linear elastic stress field data problem. We perform numerical studies to study the effect of the two end-to-end workflows and the effect of data size. Key insights and conclusions are provided, which can aid such efforts in surrogate modeling and engineering optimization.

KW - Bayesian hybrid modeling

KW - Deep neural networks

KW - Dimension reduction

KW - Image-based models

KW - Surrogate modeling

UR - http://www.scopus.com/inward/record.url?scp=85178512157&partnerID=8YFLogxK

U2 - 10.1115/DETC2023110704

DO - 10.1115/DETC2023110704

M3 - Conference contribution

AN - SCOPUS:85178512157

T3 - Proceedings of the ASME Design Engineering Technical Conference

BT - 43rd Computers and Information in Engineering Conference (CIE)

PB - American Society of Mechanical Engineers (ASME)

T2 - ASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023

Y2 - 20 August 2023 through 23 August 2023

ER -

Luan L, Ramachandra N, Ravi SK, Bhaduri A, Pandita P, Balaprakash P et al. SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS. In 43rd Computers and Information in Engineering Conference (CIE). American Society of Mechanical Engineers (ASME). 2023. v002t02a011. (Proceedings of the ASME Design Engineering Technical Conference). doi: 10.1115/DETC2023110704

SCALABLE PROBABILISTIC MODELING AND MACHINE LEARNING WITH DIMENSIONALITY REDUCTION FOR EXPENSIVE HIGH-DIMENSIONAL PROBLEMS

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this