Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis

Maciej A. Mazurowski; Piotr A. Habas; Jacek M. Zurada; Georgia D. Tourassi

doi:10.1109/IJCNN.2007.4371266

Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis

Maciej A. Mazurowski, Piotr A. Habas, Jacek M. Zurada, Georgia D. Tourassi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

This paper presents an experimental study on the impact of low class prevalence on the neural network based classifier performance as measured using Receiver Operator Characteristic (ROC) analysis. Two methods of dealing with the problem are investigated: oversampling and undersampling in the context of varying the class prevalence and the size of training datasets with uncorrelated and correlated features. The results show that the class imbalance can significantly decrease the classifier performance especially in the case of small training datasets. Furthermore, the oversampling method is shown to be more effective than the undersampling method in compensating the class imbalance. Statistically significant differences, however, are observed only in the cases with large total number of samples and very low prevalence.

Original language	English
Title of host publication	The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings
Pages	2005-2009
Number of pages	5
DOIs	https://doi.org/10.1109/IJCNN.2007.4371266
State	Published - 2007
Externally published	Yes
Event	2007 International Joint Conference on Neural Networks, IJCNN 2007 - Orlando, FL, United States Duration: Aug 12 2007 → Aug 17 2007

Publication series

Name	IEEE International Conference on Neural Networks - Conference Proceedings
ISSN (Print)	1098-7576

Conference

Conference	2007 International Joint Conference on Neural Networks, IJCNN 2007
Country/Territory	United States
City	Orlando, FL
Period	08/12/07 → 08/17/07

Access to Document

10.1109/IJCNN.2007.4371266

Cite this

Mazurowski, M. A., Habas, P. A., Zurada, J. M., & Tourassi, G. D. (2007). Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis. In The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings (pp. 2005-2009). Article 4371266 (IEEE International Conference on Neural Networks - Conference Proceedings). https://doi.org/10.1109/IJCNN.2007.4371266

Mazurowski, Maciej A. ; Habas, Piotr A. ; Zurada, Jacek M. et al. / Impact of low class prevalence on the performance evaluation of neural network based classifiers : Experimental study in the context of computer-assisted medical diagnosis. The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings. 2007. pp. 2005-2009 (IEEE International Conference on Neural Networks - Conference Proceedings).

@inproceedings{ab50cd1e7f9c48eaa7abdc92871d8630,

title = "Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis",

abstract = "This paper presents an experimental study on the impact of low class prevalence on the neural network based classifier performance as measured using Receiver Operator Characteristic (ROC) analysis. Two methods of dealing with the problem are investigated: oversampling and undersampling in the context of varying the class prevalence and the size of training datasets with uncorrelated and correlated features. The results show that the class imbalance can significantly decrease the classifier performance especially in the case of small training datasets. Furthermore, the oversampling method is shown to be more effective than the undersampling method in compensating the class imbalance. Statistically significant differences, however, are observed only in the cases with large total number of samples and very low prevalence.",

author = "Mazurowski, {Maciej A.} and Habas, {Piotr A.} and Zurada, {Jacek M.} and Tourassi, {Georgia D.}",

year = "2007",

doi = "10.1109/IJCNN.2007.4371266",

language = "English",

isbn = "142441380X",

series = "IEEE International Conference on Neural Networks - Conference Proceedings",

pages = "2005--2009",

booktitle = "The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings",

note = "2007 International Joint Conference on Neural Networks, IJCNN 2007 ; Conference date: 12-08-2007 Through 17-08-2007",

}

Mazurowski, MA, Habas, PA, Zurada, JM & Tourassi, GD 2007, Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis. in The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings., 4371266, IEEE International Conference on Neural Networks - Conference Proceedings, pp. 2005-2009, 2007 International Joint Conference on Neural Networks, IJCNN 2007, Orlando, FL, United States, 08/12/07. https://doi.org/10.1109/IJCNN.2007.4371266

Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis. / Mazurowski, Maciej A.; Habas, Piotr A.; Zurada, Jacek M. et al.
The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings. 2007. p. 2005-2009 4371266 (IEEE International Conference on Neural Networks - Conference Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Impact of low class prevalence on the performance evaluation of neural network based classifiers

T2 - 2007 International Joint Conference on Neural Networks, IJCNN 2007

AU - Mazurowski, Maciej A.

AU - Habas, Piotr A.

AU - Zurada, Jacek M.

AU - Tourassi, Georgia D.

PY - 2007

Y1 - 2007

N2 - This paper presents an experimental study on the impact of low class prevalence on the neural network based classifier performance as measured using Receiver Operator Characteristic (ROC) analysis. Two methods of dealing with the problem are investigated: oversampling and undersampling in the context of varying the class prevalence and the size of training datasets with uncorrelated and correlated features. The results show that the class imbalance can significantly decrease the classifier performance especially in the case of small training datasets. Furthermore, the oversampling method is shown to be more effective than the undersampling method in compensating the class imbalance. Statistically significant differences, however, are observed only in the cases with large total number of samples and very low prevalence.

AB - This paper presents an experimental study on the impact of low class prevalence on the neural network based classifier performance as measured using Receiver Operator Characteristic (ROC) analysis. Two methods of dealing with the problem are investigated: oversampling and undersampling in the context of varying the class prevalence and the size of training datasets with uncorrelated and correlated features. The results show that the class imbalance can significantly decrease the classifier performance especially in the case of small training datasets. Furthermore, the oversampling method is shown to be more effective than the undersampling method in compensating the class imbalance. Statistically significant differences, however, are observed only in the cases with large total number of samples and very low prevalence.

UR - http://www.scopus.com/inward/record.url?scp=51749123990&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2007.4371266

DO - 10.1109/IJCNN.2007.4371266

M3 - Conference contribution

AN - SCOPUS:51749123990

SN - 142441380X

SN - 9781424413805

T3 - IEEE International Conference on Neural Networks - Conference Proceedings

SP - 2005

EP - 2009

BT - The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings

Y2 - 12 August 2007 through 17 August 2007

ER -

Mazurowski MA, Habas PA, Zurada JM, Tourassi GD. Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis. In The 2007 International Joint Conference on Neural Networks, IJCNN 2007 Conference Proceedings. 2007. p. 2005-2009. 4371266. (IEEE International Conference on Neural Networks - Conference Proceedings). doi: 10.1109/IJCNN.2007.4371266

Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this