Video frame prediction of microbial growth with a recurrent neural network

Connor Robertson; Jared L. Wilmoth; Scott Retterer; Miguel Fuentes-Cabrera

doi:10.3389/fmicb.2022.1034586

Video frame prediction of microbial growth with a recurrent neural network

Connor Robertson, Jared L. Wilmoth, Scott Retterer, Miguel Fuentes-Cabrera

Center for Nanophase Matls Sciences

Research output: Contribution to journal › Article › peer-review

Abstract

The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.

Original language	English
Article number	1034586
Journal	Frontiers in Microbiology
Volume	13
DOIs	https://doi.org/10.3389/fmicb.2022.1034586
State	Published - Jan 5 2023

Funding

The machine learning modeling research was conducted at the Center for Nanophase Materials Sciences, which is a US Department of Energy Office of Science User Facility at Oak Ridge National Laboratory. CR was funded by U.S. Department of Energy, Office of Science, Office of Workforce Development for Teachers and Scientists, Office of Science Graduate Student Research (SCGSR) program award. The SCGSR program is administered by the Oak Ridge Institute for Science and Education (ORISE) for the DOE. ORISE is managed by ORAU under contract number DESC0014664. The authors would like to thank Andrea Timm for providing the experimental videos used for training predRNN. All the simulations were run in CADES, https://cades.ornl.gov/. The machine learning modeling research was conducted at the Center for Nanophase Materials Sciences, which is a US Department of Energy Office of Science User Facility at Oak Ridge National Laboratory. CR was funded by U.S. Department of Energy, Office of Science, Office of Workforce Development for Teachers and Scientists, Office of Science Graduate Student Research (SCGSR) program award. The SCGSR program is administered by the Oak Ridge Institute for Science and Education (ORISE) for the DOE. ORISE is managed by ORAU under contract number DESC0014664. The authors would like to thank Andrea Timm for providing the experimental videos used for training predRNN. All the simulations were run in CADES, https://cades.ornl.gov/ . This article has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this article, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan ( http://energy.gov/downloads/doe-public-access-plan ).

Funders	Funder number
Office of Science Graduate Student Research
SCGSR
U.S. Department of Energy
Office of Science
Workforce Development for Teachers and Scientists
Oak Ridge Associated Universities	DESC0014664
Oak Ridge National Laboratory
Oak Ridge Institute for Science and Education

Keywords

Recurrent Neural Network
deep learning
machine learning
microbial growth
population growth
video frame prediction

Access to Document

10.3389/fmicb.2022.1034586

Cite this

@article{682fa09338e1499795cfc112b1bea5ad,

title = "Video frame prediction of microbial growth with a recurrent neural network",

abstract = "The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.",

keywords = "Recurrent Neural Network, deep learning, machine learning, microbial growth, population growth, video frame prediction",

author = "Connor Robertson and Wilmoth, {Jared L.} and Scott Retterer and Miguel Fuentes-Cabrera",

note = "Publisher Copyright: Copyright {\textcopyright} 2023 Robertson, Wilmoth, Retterer and Fuentes-Cabrera.",

year = "2023",

month = jan,

day = "5",

doi = "10.3389/fmicb.2022.1034586",

language = "English",

volume = "13",

journal = "Frontiers in Microbiology",

issn = "1664-302X",

publisher = "Frontiers Media",

}

TY - JOUR

T1 - Video frame prediction of microbial growth with a recurrent neural network

AU - Robertson, Connor

AU - Wilmoth, Jared L.

AU - Retterer, Scott

AU - Fuentes-Cabrera, Miguel

PY - 2023/1/5

Y1 - 2023/1/5

N2 - The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.

AB - The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.

KW - Recurrent Neural Network

KW - deep learning

KW - machine learning

KW - microbial growth

KW - population growth

KW - video frame prediction

UR - http://www.scopus.com/inward/record.url?scp=85146496471&partnerID=8YFLogxK

U2 - 10.3389/fmicb.2022.1034586

DO - 10.3389/fmicb.2022.1034586

M3 - Article

AN - SCOPUS:85146496471

SN - 1664-302X

VL - 13

JO - Frontiers in Microbiology

JF - Frontiers in Microbiology

M1 - 1034586

ER -

Video frame prediction of microbial growth with a recurrent neural network

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this