TY - JOUR
T1 - Physics and chemistry from parsimonious representations
T2 - image analysis via invariant variational autoencoders
AU - Valleti, Mani
AU - Ziatdinov, Maxim
AU - Liu, Yongtao
AU - Kalinin, Sergei V.
N1 - Publisher Copyright:
© UT-Battelle,LLC, Battelle Memorial Institute and Mani Valleti 2024.
PY - 2024/12
Y1 - 2024/12
N2 - Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenomena from the data, such as manifestations of symmetry breaking phenomena in electron and scanning tunneling microscopy images, or variability of the nanoparticles. Variational autoencoders (VAEs) are emerging as a powerful paradigm for the unsupervised data analysis, allowing to disentangle the factors of variability and discover optimal parsimonious representation. Here, we summarize recent developments in VAEs, covering the basic principles and intuition behind the VAEs. The invariant VAEs are introduced as an approach to accommodate scale and translation invariances present in imaging data and separate known factors of variations from the ones to be discovered. We further describe the opportunities enabled by the control over VAE architecture, including conditional, semi-supervised, and joint VAEs. Several case studies of VAE applications for toy models and experimental datasets in Scanning Transmission Electron Microscopy are discussed, emphasizing the deep connection between VAE and basic physical principles. Python codes and datasets discussed in this article are available at https://github.com/saimani5/VAE-tutorials and can be used by researchers as an application guide when applying these to their own datasets.
AB - Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenomena from the data, such as manifestations of symmetry breaking phenomena in electron and scanning tunneling microscopy images, or variability of the nanoparticles. Variational autoencoders (VAEs) are emerging as a powerful paradigm for the unsupervised data analysis, allowing to disentangle the factors of variability and discover optimal parsimonious representation. Here, we summarize recent developments in VAEs, covering the basic principles and intuition behind the VAEs. The invariant VAEs are introduced as an approach to accommodate scale and translation invariances present in imaging data and separate known factors of variations from the ones to be discovered. We further describe the opportunities enabled by the control over VAE architecture, including conditional, semi-supervised, and joint VAEs. Several case studies of VAE applications for toy models and experimental datasets in Scanning Transmission Electron Microscopy are discussed, emphasizing the deep connection between VAE and basic physical principles. Python codes and datasets discussed in this article are available at https://github.com/saimani5/VAE-tutorials and can be used by researchers as an application guide when applying these to their own datasets.
UR - http://www.scopus.com/inward/record.url?scp=85201293616&partnerID=8YFLogxK
U2 - 10.1038/s41524-024-01250-5
DO - 10.1038/s41524-024-01250-5
M3 - Article
AN - SCOPUS:85201293616
SN - 2057-3960
VL - 10
JO - npj Computational Materials
JF - npj Computational Materials
IS - 1
M1 - 183
ER -