A search engine for remote database-aided interpretation of digitized mammograms

We describe a query-by-content search engine that enables a radiologist to search a large database of diagnosticallyproven ("benign" or "malignant") manimographic regions of interest (ROIs). The database search is facilitated by a relational map which is a two-dimensional display ofall the ROIs in the database. Labeled points on the map represent ROTs in the database. The map is constructed from the output of a neural network that has been trained to cluster the ROTs in the database using a measure of perceptual similarity. To use the search facility of our computer-aided diagnosis system a radiologist selects a ROT from a digitized manimogram and submits the ROI as a query to the search engine. The search engine first maps the query ROT to its appropriate location on the relational map. The search engine then retrieves the ROTs that are closest to the query ROT on the relational map. These retrieved ROTs are from the same cluster on the relational map. The results of the search are presented to the radiologist in the form of the retrieved ROTs, along with related information such as biopsy result and patient age. The radiologist can also perform an unrestricted search by selecting any point on the relational map. The search engine will then return the closest ROTs to the selected point. The search engine is implemented using a three-layer distributed architecture. The first layer is a Java-based user interface that allows a radiologist to view a digital manimogram, to enhance the mammogram, to select a ROT, and to query the database. The second layer is a web server that generates HTML for the web client and provides access to the image processing algorithms, the neural network, and the image search functions. The third layer is a remote database containing the ROTs and associated patient information. The embedding of this search engine into an integrated diagnostic system may help the radiologist to incorporate subtle image relationships into the diagnostic process, which in turn may lead to improved diagnostic accuracy.