Inverse dependence of search and classification performances in lesion localization tasks

D. P. Chakraborty, Hong Jun Yoon, Claudia Mello-Thoms

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Search involves detecting the locations of potential lesions. Classification involves determining if a detected region is a true lesion. The most commonly used measure of observer performance, namely the area A under the ROC curve, is affected by both search and classification performances. The aim was to demonstrate a method for separating these contributions and apply it to several clinical datasets. Search performance S was defined as the square root of 2 times the perpendicular distance of the end-point of the search-model predicted ROC from the chance diagonal. Classification performance C was defined as the separation of the unit-variance binormal distributions for signal and noise sites. Eleven (11) datasets were fitted by the search model and search, classification and trapezoidal A were computed for each modality and reader combination. Kendall-tau correlations were calculated between the resulting S, C and A pairs. Kendall correlation (S vs. C) was smaller than zero for all datasets, and the average Kendall correlation was significantly smaller than 0 (average = -0.401, P = 8.3 x 10-6). Also, Kendall correlation (A vs. S) was larger than zero for 9 out of 11 datasets and the average Kendall correlation was significantly larger than 0 (average = 0.295, P = 2.9 x 10-3). On the other hand average Kendall correlation (A vs. C) was not significantly different from zero (average = 0.102, P = 0.25). The results suggest that radiologists may learn to compensate for poor search performance with better classification performance. This study also indicates that efforts at improving net performance, which currently focus almost exclusively on improving classification performance, may be more successful if aimed at improving search performance.

Original languageEnglish
Title of host publicationMedical Imaging 2012
Subtitle of host publicationImage Perception, Observer Performance, and Technology Assessment
EditorsCraig K. Abbey, Claudia R. Mello-Thoms
PublisherSPIE
ISBN (Electronic)9780819489678
DOIs
StatePublished - 2012
Externally publishedYes
EventMedical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment - San Diego, United States
Duration: Feb 8 2012Feb 9 2012

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume8318
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment
Country/TerritoryUnited States
CitySan Diego
Period02/8/1202/9/12

Funding

Acknowledgments: DPC and CMT were supported in part by grants from the Department of Health and Human Services, National Institutes of Health, R01-EB005243 and R01-EB008688. We would like to thank the following who contributed datasets for this study: Dr. S. Taylor-Phillips, Dr. Federica Zanca, Dr. Tony Svahn, Dr. Sara Asplund, Dr. Magnus Bath, Ms. Lucy Warren, Dr. Nico Buls, Dr. Monica Penedo, Dr. Kevin Berbaum and Dr. Mark Ruschin.

Keywords

  • Classification
  • Correlations
  • FROC
  • Observer performance
  • ROC
  • Search
  • Search-model

Fingerprint

Dive into the research topics of 'Inverse dependence of search and classification performances in lesion localization tasks'. Together they form a unique fingerprint.

Cite this