Abstract
The proliferation of optical, electron, and scanning probe microscopies gives rise to large volumes of imaging data of objects as diversified as cells, bacteria, and pollen, to nanoparticles and atoms and molecules. In most cases, the experimental data streams contain images having arbitrary rotations and translations within the image. At the same time, for many cases, small amounts of labeled data are available in the form of prior published results, image collections, and catalogs, or even theoretical models. Here we develop an approach that allows generalizing from a small subset of labeled data with a weak orientational disorder to a large unlabeled dataset with a much stronger orientational (and positional) disorder, i.e., it performs a classification of image data given a small number of examples even in the presence of a distribution shift between the labeled and unlabeled parts. This approach is based on the semi-supervised rotationally invariant variational autoencoder (ss-rVAE) model consisting of the encoder-decoder “block” that learns a rotationally-invariant latent representation of data and a classifier for categorizing data into different discrete classes. The classifier part of the trained ss-rVAE inherits the rotational (and translational) invariances and can be deployed independently of the other parts of the model. The performance of the ss-rVAE is illustrated using the synthetic data sets with known factors of variation. We further demonstrate its application for experimental data sets of nanoparticles, creating nanoparticle libraries and disentangling the representations defining the physical factors of variation in the data.
| Original language | English |
|---|---|
| Pages (from-to) | 1213-1220 |
| Number of pages | 8 |
| Journal | Digital Discovery |
| Volume | 3 |
| Issue number | 6 |
| DOIs | |
| State | Published - May 21 2024 |
Funding
This work was supported (M. Y. Y., D. G., S. V. K.) by the US Department of Energy, Office of Science, Office of Basic Energy Sciences, as part of the Energy Frontier Research Centers program: CSSAS \u2013 The Center for the Science of Synthesis Across Scales \u2013 under award number DE-SC0019288, located at University of Washington. SEM imaging was conducted at the University of Washington Molecular Analysis Facility, a National Nanotechnology Coordinated Infrastructure (NNCI) site which is supported in part by the National Science Foundation, the University of Washington, the Molecular Engineering and Sciences Institute, and the Clean Energy Institute. D. S. G. acknowledges support from the University of Washington, Department of Chemistry Kwiram Endowment. The ML of experimental data is supported (Y. L.) by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences Energy Frontier Research Centers program under award number DE-SC0021118. The development and maintenance of the pyroVED software is supported (M. Z.) by the Laboratory Directed Research and Development Program at Pacific Northwest National Laboratory, a multiprogram national laboratory operated by Battelle for the U.S. Department of Energy. The authors acknowledge Dr Ilia Ivanov (CNMS) for early ideas (\u223C2010) of creating nanoparticle libraries.