Explainability and human intervention in autonomous scanning probe microscopy

Yongtao Liu; Maxim A. Ziatdinov; Rama K. Vasudevan; Sergei V. Kalinin

doi:10.1016/j.patter.2023.100858

Explainability and human intervention in autonomous scanning probe microscopy

Yongtao Liu, Maxim A. Ziatdinov, Rama K. Vasudevan, Sergei V. Kalinin

Data Nanoanalytics

Research output: Contribution to journal › Article › peer-review

5 Scopus citations

Abstract

The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.

Original language	English
Article number	100858
Journal	Patterns
Volume	4
Issue number	11
DOIs	https://doi.org/10.1016/j.patter.2023.100858
State	Published - Nov 10 2023

Funding

This effort (SPM measurement, post-experimental analysis workflow development, data analysis) was primarily supported by the Center for 3D Ferroelectric Microelectronics (3DFeM), an Energy Frontier Research Center funded by the US Department of Energy (DOE), Office of Science, Basic Energy Sciences under award number DE-SC0021118. SPM experiments were done at the Center for Nanophase Materials Sciences (CNMS), which is a US Department of Energy, Office of Science User Facility at Oak Ridge National Laboratory. S.V.K. acknowledges support from the Center for Nanophase Materials Sciences (CNMS) user facility, which is a US Department of Energy, Office of Science User Facility, user project no. CNMS2022-B-01642. S.V.K. R.K.V. M.A.Z. and Y.L. conceived the project. Y.L. developed post-experimental workflow based on DKL and rVAE from M.A.Z. All authors contributed to discussions and the final manuscript. The authors declare no conflict of interest. This effort (SPM measurement, post-experimental analysis workflow development, data analysis) was primarily supported by the Center for 3D Ferroelectric Microelectronics (3DFeM), an Energy Frontier Research Center funded by the US Department of Energy (DOE), Office of Science , Basic Energy Sciences under award number DE-SC0021118 . SPM experiments were done at the Center for Nanophase Materials Sciences (CNMS), which is a US Department of Energy , Office of Science User Facility at Oak Ridge National Laboratory . S.V.K. acknowledges support from the Center for Nanophase Materials Sciences (CNMS) user facility, which is a US Department of Energy, Office of Science User Facility , user project no. CNMS2022-B-01642 .

Funders	Funder number
center for 3D Ferroelectric Microelectronics
Center for Nanophase Materials Sciences	CNMS2022-B-01642
U.S. Department of Energy
Office of Science
Basic Energy Sciences	DE-SC0021118
Oak Ridge National Laboratory

Keywords

DSML 2: Proof-of-Concept: Data science output has been formulated, implemented, and tested for one domain/problem
Gaussian process
autonomous experiments
deep kernel learning
human in the loop
scanning probe microscopy

Access to Document

10.1016/j.patter.2023.100858

Cite this

@article{3edf628a941e42ecad3a57b097523f1d,

title = "Explainability and human intervention in autonomous scanning probe microscopy",

abstract = "The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.",

keywords = "DSML 2: Proof-of-Concept: Data science output has been formulated, implemented, and tested for one domain/problem, Gaussian process, autonomous experiments, deep kernel learning, human in the loop, scanning probe microscopy",

author = "Yongtao Liu and Ziatdinov, {Maxim A.} and Vasudevan, {Rama K.} and Kalinin, {Sergei V.}",

note = "Publisher Copyright: {\textcopyright} 2023 The Author(s), Oak Ridge National Laboratory",

year = "2023",

month = nov,

day = "10",

doi = "10.1016/j.patter.2023.100858",

language = "English",

volume = "4",

journal = "Patterns",

issn = "2666-3899",

publisher = "Cell Press",

number = "11",

}

TY - JOUR

T1 - Explainability and human intervention in autonomous scanning probe microscopy

AU - Liu, Yongtao

AU - Ziatdinov, Maxim A.

AU - Vasudevan, Rama K.

AU - Kalinin, Sergei V.

PY - 2023/11/10

Y1 - 2023/11/10

N2 - The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.

AB - The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.

KW - DSML 2: Proof-of-Concept: Data science output has been formulated, implemented, and tested for one domain/problem

KW - Gaussian process

KW - autonomous experiments

KW - deep kernel learning

KW - human in the loop

KW - scanning probe microscopy

UR - http://www.scopus.com/inward/record.url?scp=85175616723&partnerID=8YFLogxK

U2 - 10.1016/j.patter.2023.100858

DO - 10.1016/j.patter.2023.100858

M3 - Article

AN - SCOPUS:85175616723

SN - 2666-3899

VL - 4

JO - Patterns

JF - Patterns

IS - 11

M1 - 100858

ER -

Explainability and human intervention in autonomous scanning probe microscopy

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this