Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification

Daniel S. Adams; Taylor Hauser; H. Lexie Yang; Peter Li

doi:10.1145/3687123.3698282

Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification

Daniel S. Adams, Taylor Hauser, H. Lexie Yang, Peter Li

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This study addresses the challenges inherent in building use type classification, particularly focusing on the issue of class imbalance in the training datasets for machine learning classifiers. We comprehensively analyze the efficacy of various class-balancing sampling techniques. Employing Monte Carlo simulations and Bayesian optimization, we evaluated the performance of multiple sampling methods, including Random Oversampling, Random Undersampling, SMOTE, Borderline-SMOTE, and ADASYN, across a dataset encompassing nine southeastern coastal states of the United States. Our findings reveal that simple random over-and undersampling techniques outperform more sophisticated methods. Additionally, we show inherent value in creating an imbalance in training data to effectively train a machine learning classifier for distinguishing between residential and nonresidential buildings. This study provides valuable guidance for future research on building use type classification research and lays essential groundwork for developing attribute-rich building stock datasets.

Original language	English
Title of host publication	GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery
Editors	Song Gao, Gengchen Mai, Shawn Newsam, Lexie Yang, Dalton Lunga, Di Zhu, Bruno Martins, Samantha Arundel
Publisher	Association for Computing Machinery, Inc
Pages	15-26
Number of pages	12
ISBN (Electronic)	9798400711763
DOIs	https://doi.org/10.1145/3687123.3698282
State	Published - Nov 18 2024
Event	7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, GeoAI 2024 - Atlanta, United States Duration: Oct 29 2024 → …

Publication series

Name	GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery

Conference

Conference	7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, GeoAI 2024
Country/Territory	United States
City	Atlanta
Period	10/29/24 → …

Funding

This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-publicaccess-plan).

Keywords

Building Use Type
Class Imbalance
Machine Learning
Monte Carlo
Sampling Techniques

Access to Document

10.1145/3687123.3698282

Cite this

Adams, D. S., Hauser, T., Yang, H. L., & Li, P. (2024). Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification. In S. Gao, G. Mai, S. Newsam, L. Yang, D. Lunga, D. Zhu, B. Martins, & S. Arundel (Eds.), GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery (pp. 15-26). (GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery). Association for Computing Machinery, Inc. https://doi.org/10.1145/3687123.3698282

Adams, Daniel S. ; Hauser, Taylor ; Yang, H. Lexie et al. / Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification. GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. editor / Song Gao ; Gengchen Mai ; Shawn Newsam ; Lexie Yang ; Dalton Lunga ; Di Zhu ; Bruno Martins ; Samantha Arundel. Association for Computing Machinery, Inc, 2024. pp. 15-26 (GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery).

@inproceedings{d817b8e356da42f4a9e20a84ceaa3bfc,

title = "Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification",

abstract = "This study addresses the challenges inherent in building use type classification, particularly focusing on the issue of class imbalance in the training datasets for machine learning classifiers. We comprehensively analyze the efficacy of various class-balancing sampling techniques. Employing Monte Carlo simulations and Bayesian optimization, we evaluated the performance of multiple sampling methods, including Random Oversampling, Random Undersampling, SMOTE, Borderline-SMOTE, and ADASYN, across a dataset encompassing nine southeastern coastal states of the United States. Our findings reveal that simple random over-and undersampling techniques outperform more sophisticated methods. Additionally, we show inherent value in creating an imbalance in training data to effectively train a machine learning classifier for distinguishing between residential and nonresidential buildings. This study provides valuable guidance for future research on building use type classification research and lays essential groundwork for developing attribute-rich building stock datasets.",

keywords = "Building Use Type, Class Imbalance, Machine Learning, Monte Carlo, Sampling Techniques",

author = "Adams, \{Daniel S.\} and Taylor Hauser and Yang, \{H. Lexie\} and Peter Li",

note = "Publisher Copyright: {\textcopyright} 2024 Copyright held by the owner/author(s).; 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, GeoAI 2024 ; Conference date: 29-10-2024",

year = "2024",

month = nov,

day = "18",

doi = "10.1145/3687123.3698282",

language = "English",

series = "GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery",

publisher = "Association for Computing Machinery, Inc",

pages = "15--26",

editor = "Song Gao and Gengchen Mai and Shawn Newsam and Lexie Yang and Dalton Lunga and Di Zhu and Bruno Martins and Samantha Arundel",

booktitle = "GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery",

}

Adams, DS, Hauser, T, Yang, HL & Li, P 2024, Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification. in S Gao, G Mai, S Newsam, L Yang, D Lunga, D Zhu, B Martins & S Arundel (eds), GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, Association for Computing Machinery, Inc, pp. 15-26, 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, GeoAI 2024, Atlanta, United States, 10/29/24. https://doi.org/10.1145/3687123.3698282

Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification. / Adams, Daniel S.; Hauser, Taylor; Yang, H. Lexie et al.
GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. ed. / Song Gao; Gengchen Mai; Shawn Newsam; Lexie Yang; Dalton Lunga; Di Zhu; Bruno Martins; Samantha Arundel. Association for Computing Machinery, Inc, 2024. p. 15-26 (GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification

AU - Adams, Daniel S.

AU - Hauser, Taylor

AU - Yang, H. Lexie

AU - Li, Peter

PY - 2024/11/18

Y1 - 2024/11/18

N2 - This study addresses the challenges inherent in building use type classification, particularly focusing on the issue of class imbalance in the training datasets for machine learning classifiers. We comprehensively analyze the efficacy of various class-balancing sampling techniques. Employing Monte Carlo simulations and Bayesian optimization, we evaluated the performance of multiple sampling methods, including Random Oversampling, Random Undersampling, SMOTE, Borderline-SMOTE, and ADASYN, across a dataset encompassing nine southeastern coastal states of the United States. Our findings reveal that simple random over-and undersampling techniques outperform more sophisticated methods. Additionally, we show inherent value in creating an imbalance in training data to effectively train a machine learning classifier for distinguishing between residential and nonresidential buildings. This study provides valuable guidance for future research on building use type classification research and lays essential groundwork for developing attribute-rich building stock datasets.

AB - This study addresses the challenges inherent in building use type classification, particularly focusing on the issue of class imbalance in the training datasets for machine learning classifiers. We comprehensively analyze the efficacy of various class-balancing sampling techniques. Employing Monte Carlo simulations and Bayesian optimization, we evaluated the performance of multiple sampling methods, including Random Oversampling, Random Undersampling, SMOTE, Borderline-SMOTE, and ADASYN, across a dataset encompassing nine southeastern coastal states of the United States. Our findings reveal that simple random over-and undersampling techniques outperform more sophisticated methods. Additionally, we show inherent value in creating an imbalance in training data to effectively train a machine learning classifier for distinguishing between residential and nonresidential buildings. This study provides valuable guidance for future research on building use type classification research and lays essential groundwork for developing attribute-rich building stock datasets.

KW - Building Use Type

KW - Class Imbalance

KW - Machine Learning

KW - Monte Carlo

KW - Sampling Techniques

UR - https://www.scopus.com/pages/publications/85215089506

U2 - 10.1145/3687123.3698282

DO - 10.1145/3687123.3698282

M3 - Conference contribution

AN - SCOPUS:85215089506

T3 - GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery

SP - 15

EP - 26

BT - GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery

A2 - Gao, Song

A2 - Mai, Gengchen

A2 - Newsam, Shawn

A2 - Yang, Lexie

A2 - Lunga, Dalton

A2 - Zhu, Di

A2 - Martins, Bruno

A2 - Arundel, Samantha

PB - Association for Computing Machinery, Inc

T2 - 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, GeoAI 2024

Y2 - 29 October 2024

ER -

Adams DS, Hauser T, Yang HL, Li P. Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification. In Gao S, Mai G, Newsam S, Yang L, Lunga D, Zhu D, Martins B, Arundel S, editors, GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. Association for Computing Machinery, Inc. 2024. p. 15-26. (GeoAI 2024 - Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery). doi: 10.1145/3687123.3698282

Multivariate Testing of Sampling Techniques to Address Class Imbalance in Building Use Type Classification

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this