Confidence-based stratification of CAD recommendations with application to breast cancer detection

We present a risk stratification methodology for predictions made by computer-assisted detection (CAD) systems. For each positive CAD prediction, the proposed technique assigns an individualized confidence measure as a function of the actual CAD output, the case-specific uncertainty of the prediction estimated from the system's performance for similar cases and the value of the operating decision threshold. The study was performed using a mammographic database containing 1,337 regions of interest (ROIs) with known ground truth (681 with masses, 656 with normal parenchyma). Two types of decision models (1) a support vector machine (SVM) with a radial basis function kernel and (2) a back-propagation neural network (BPNN) were developed to detect masses based on 8 morphological features automatically extracted from each ROI. The study shows that as requirements on the minimum confidence value are being restricted, the positive predictive value (PPV) for qualifying cases steadily improves (from PPV = 0.73 to PPV = 0.97 for the SVM, from PPV = 0.67 to PPV = 0.95 for the BPNN). The proposed confidence metric was successfully applied for stratification of CAD recommendations into 3 categories of different expected reliability: HIGH (PPV = 0.90), LOW (PPV = 0.30) and MEDIUM (all remaining cases). Since radiologists often disregard accurate CAD cues, an individualized confidence measure should improve their ability to correctly process visual cues and thus reduce the interpretation error associated with the detection task. While keeping the clinically determined operating point satisfied, the proposed methodology draws the CAD users' attention to cases/regions of highest risk while helping them confidently eliminate cases with low risk.