YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Amirkoushyar Ziabari; Derek C. Rose; Abbas Shirinifard; David Solecki

doi:10.1016/j.patrec.2024.03.015

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Amirkoushyar Ziabari
, Derek C. Rose
, Abbas Shirinifard
, David Solecki

Multimodal Sensor Analytics

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. A comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced here. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

Original language	English
Pages (from-to)	37-42
Number of pages	6
Journal	Pattern Recognition Letters
Volume	181
DOIs	https://doi.org/10.1016/j.patrec.2024.03.015
State	Published - May 2024

Funding

Research sponsored by the U.S. Department of Energy, under contract DE-AC05-00OR22725 with UT-Battelle, LLC. The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). This collaborations was funded by St. Jude Children's Research Hospital through funding from the American Lebanese Syrian Associated Charities (ALSAC). The Solecki Laboratory is funded by grants 1R01NS066936 and R01NS104029-02 from the National Institute of Neurological Disorders (NINDS). Research sponsored by the U.S. Department of Energy , under contract DE-AC05-00OR22725 with UT-Battelle, LLC. The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan ( http://energy.gov/downloads/doe-public-access-plan ). A.D and S.V were supported by AI-initiative LDRD Program at ORNL . This collaborations was funded by St. Jude Children’s Research Hospital through funding from the American Lebanese Syrian Associated Charities (ALSAC). The Solecki Laboratory is funded by grants 1R01NS066936 and R01NS104029-02 from the National Institute of Neurological Disorders (NINDS).

Keywords

3D instance segmentation
Cell microscopy
Deep learning

Access to Document

10.1016/j.patrec.2024.03.015

Cite this

@article{e18d24c7f1b3487dac7f73b55e7f4486,

title = "YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy",

abstract = "Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. A comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced here. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.",

keywords = "3D instance segmentation, Cell microscopy, Deep learning",

author = "Amirkoushyar Ziabari and Rose, \{Derek C.\} and Abbas Shirinifard and David Solecki",

note = "Publisher Copyright: {\textcopyright} 2024",

year = "2024",

month = may,

doi = "10.1016/j.patrec.2024.03.015",

language = "English",

volume = "181",

pages = "37--42",

journal = "Pattern Recognition Letters",

issn = "0167-8655",

publisher = "Elsevier",

}

TY - JOUR

T1 - YOLO2U-Net

T2 - Detection-guided 3D instance segmentation for microscopy

AU - Ziabari, Amirkoushyar

AU - Rose, Derek C.

AU - Shirinifard, Abbas

AU - Solecki, David

PY - 2024/5

Y1 - 2024/5

N2 - Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. A comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced here. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

AB - Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. A comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced here. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

KW - 3D instance segmentation

KW - Cell microscopy

KW - Deep learning

UR - https://www.scopus.com/pages/publications/85188780213

U2 - 10.1016/j.patrec.2024.03.015

DO - 10.1016/j.patrec.2024.03.015

M3 - Article

AN - SCOPUS:85188780213

SN - 0167-8655

VL - 181

SP - 37

EP - 42

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

ER -

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this