Abstract
Search involves detecting the locations of potential lesions. Classification involves determining if a detected region is a true lesion. The most commonly used measure of observer performance, namely the area A under the ROC curve, is affected by both search and classification performances. The aim was to demonstrate a method for separating these contributions and apply it to several clinical datasets. Search performance S was defined as the square root of 2 times the perpendicular distance of the end-point of the search-model predicted ROC from the chance diagonal. Classification performance C was defined as the separation of the unit-variance binormal distributions for signal and noise sites. Eleven (11) datasets were fitted by the search model and search, classification and trapezoidal A were computed for each modality and reader combination. Kendall-tau correlations were calculated between the resulting S, C and A pairs. Kendall correlation (S vs. C) was smaller than zero for all datasets, and the average Kendall correlation was significantly smaller than 0 (average = -0.401, P = 8.3 x 10-6). Also, Kendall correlation (A vs. S) was larger than zero for 9 out of 11 datasets and the average Kendall correlation was significantly larger than 0 (average = 0.295, P = 2.9 x 10-3). On the other hand average Kendall correlation (A vs. C) was not significantly different from zero (average = 0.102, P = 0.25). The results suggest that radiologists may learn to compensate for poor search performance with better classification performance. This study also indicates that efforts at improving net performance, which currently focus almost exclusively on improving classification performance, may be more successful if aimed at improving search performance.
Original language | English |
---|---|
Title of host publication | Medical Imaging 2012 |
Subtitle of host publication | Image Perception, Observer Performance, and Technology Assessment |
Editors | Craig K. Abbey, Claudia R. Mello-Thoms |
Publisher | SPIE |
ISBN (Electronic) | 9780819489678 |
DOIs | |
State | Published - 2012 |
Externally published | Yes |
Event | Medical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment - San Diego, United States Duration: Feb 8 2012 → Feb 9 2012 |
Publication series
Name | Progress in Biomedical Optics and Imaging - Proceedings of SPIE |
---|---|
Volume | 8318 |
ISSN (Print) | 1605-7422 |
Conference
Conference | Medical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment |
---|---|
Country/Territory | United States |
City | San Diego |
Period | 02/8/12 → 02/9/12 |
Funding
Acknowledgments: DPC and CMT were supported in part by grants from the Department of Health and Human Services, National Institutes of Health, R01-EB005243 and R01-EB008688. We would like to thank the following who contributed datasets for this study: Dr. S. Taylor-Phillips, Dr. Federica Zanca, Dr. Tony Svahn, Dr. Sara Asplund, Dr. Magnus Bath, Ms. Lucy Warren, Dr. Nico Buls, Dr. Monica Penedo, Dr. Kevin Berbaum and Dr. Mark Ruschin.
Keywords
- Classification
- Correlations
- FROC
- Observer performance
- ROC
- Search
- Search-model