Machine learning in scanning transmission electron microscopy

Sergei V. Kalinin, Colin Ophus, Paul M. Voyles, Rolf Erni, Demie Kepaptsoglou, Vincenzo Grillo, Andrew R. Lupini, Mark P. Oxley, Eric Schwenker, Maria K.Y. Chan, Joanne Etheridge, Xiang Li, Grace G.D. Han, Maxim Ziatdinov, Naoya Shibata, Stephen J. Pennycook

Research output: Contribution to journalReview articlepeer-review

63 Scopus citations

Abstract

Scanning transmission electron microscopy (STEM) has emerged as a uniquely powerful tool for structural and functional imaging of materials on the atomic level. Driven by advances in aberration correction, STEM now allows the routine imaging of structures with single-digit picometre-level precision for localization of atomic units. This Primer focuses on the opportunities emerging at the interface between STEM and machine learning (ML) methods. We review the primary STEM imaging methods, including structural imaging, electron energy loss spectroscopy and its momentum-resolved modalities and 4D-STEM. We discuss the quantification of STEM structural data as a necessary step towards meaningful ML applications and its analysis in terms of the relevant physics and chemistry. We show examples of the opportunities offered by structural STEM imaging in elucidating the chemistry and physics of complex materials and how the latter connect to first-principles and phase-field models to yield consistent interpretation of generative physics. We present the critical infrastructural needs for the broad adoption of ML methods in the STEM community, including the storage of data and metadata to allow the reproduction of experiments. Finally, we discuss the application of ML to automating experiments and novel scanning modes.

Original languageEnglish
Article number11
JournalNature Reviews Methods Primers
Volume2
Issue number1
DOIs
StatePublished - Dec 2022

Funding

This work is based upon work supported by the US Department of Energy (DOE), Office of Science, Basic Energy Sciences (BES), Materials Sciences and Engineering Division (S.V.K., M.P.O. and A.R.L.) and was performed and partially supported (M.Z.) at Oak Ridge National Laboratory’s Center for Nanophase Materials Sciences (CNMS), a US Department of Energy, Office of Science User Facility. V.G. acknowledges the support of the European Union Horizon 2020 Research and Innovation Programme under grant agreement no. 766970 Q-SORT (H2020-FETOPEN-1-2016-2017), no. 964591 Smart-electrons (H2020-FETOPEN-1-2016-2017) and no. 101035013 MINEON (H2020-FETOPEN-03-2018-2019-2020 – FET Innovation Launchpad). SuperSTEM (D.K.) is the UK National Facility for Advanced Electron Microscopy funded by the Engineering and Physical Sciences Research Council (EPSRC). P.M.V. acknowledges support from DOE BES (DE-FG02-08ER46547). G.G.D.H. and X.L. acknowledge support from Brandeis NSF MRSEC, Bioinspired Soft Materials, DMR-2011846. Work at the Molecular Foundry (C.O.) was supported by the Office of Science, Office of Basic Energy Sciences, of the US Department of Energy under contract no. DE-AC02-05CH11231. E.S. and M.K.Y.C. acknowledges support from the Center for Nanoscale Materials (DOE SUF) under contract no. DE-AC02-06CH11357. C.O. and M.K.Y.C. acknowledge support from DOE Early Career Research Awards. N.S. acknowledges support from the JSPS KAKENHI (grants 20H05659 and 19H05788). J.E. acknowledges the support of the Australian Research Council Discovery Project (grants DP150104483 and DP160104679). The authors are extremely grateful to K. More (from CNMS) for careful reading and editing of the manuscript.

FundersFunder number
Bioinspired Soft MaterialsDMR-2011846
Brandeis NSF MRSEC
CNMS
Center for Nanoscale Materials
DOE SUFDE-AC02-06CH11357
Oak Ridge National Laboratory
U.S. Department of Energy
Office of Science
Basic Energy SciencesDE-AC02-05CH11231, DE-FG02-08ER46547
Division of Materials Sciences and Engineering
Engineering and Physical Sciences Research Council
Australian Research CouncilDP160104679, DP150104483
Japan Society for the Promotion of Science19H05788, 20H05659
Horizon 2020101035013, 964591, 766970 Q-SORT, H2020-FETOPEN-1-2016-2017

    Fingerprint

    Dive into the research topics of 'Machine learning in scanning transmission electron microscopy'. Together they form a unique fingerprint.

    Cite this