Bias in Reinforcement Learning: A Review in Healthcare Applications

Benjamin Smith; Anahita Khojandi; Rama Vasudevan

doi:10.1145/3609502

Bias in Reinforcement Learning: A Review in Healthcare Applications

Benjamin Smith, Anahita Khojandi, Rama Vasudevan

Data Nanoanalytics

Research output: Contribution to journal › Article › peer-review

7 Scopus citations

Abstract

Reinforcement learning (RL) can assist in medical decision making using patient data collected in electronic health record (EHR) systems. RL, a type of machine learning, can use these data to develop treatment policies. However, RL models are typically trained using imperfect retrospective EHR data. Therefore, if care is not taken in training, RL policies can propagate existing bias in healthcare. Literature that considers and addresses the issues of bias and fairness in sequential decision making are reviewed. The major themes to mitigate bias that emerge relate to (1) data management; (2) algorithmic design; and (3) clinical understanding of the resulting policies.

Original language	English
Article number	52
Journal	ACM Computing Surveys
Volume	56
Issue number	2
DOIs	https://doi.org/10.1145/3609502
State	Published - Sep 15 2023

Funding

The research is partially supported by Science Alliance, The University of Tennessee, and the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the U.S. Department of Energy.

Keywords

Reinforcement learning
algorithmic bias
bias management
electronic health records
treatment planning

Access to Document

10.1145/3609502

Cite this

@article{0b3c74e2087f494698af01356d15e72b,

title = "Bias in Reinforcement Learning: A Review in Healthcare Applications",

abstract = "Reinforcement learning (RL) can assist in medical decision making using patient data collected in electronic health record (EHR) systems. RL, a type of machine learning, can use these data to develop treatment policies. However, RL models are typically trained using imperfect retrospective EHR data. Therefore, if care is not taken in training, RL policies can propagate existing bias in healthcare. Literature that considers and addresses the issues of bias and fairness in sequential decision making are reviewed. The major themes to mitigate bias that emerge relate to (1) data management; (2) algorithmic design; and (3) clinical understanding of the resulting policies.",

keywords = "Reinforcement learning, algorithmic bias, bias management, electronic health records, treatment planning",

author = "Benjamin Smith and Anahita Khojandi and Rama Vasudevan",

note = "Publisher Copyright: Copyright {\textcopyright} 2023 held by the owner/author(s). Publication rights licensed to ACM.",

year = "2023",

month = sep,

day = "15",

doi = "10.1145/3609502",

language = "English",

volume = "56",

journal = "ACM Computing Surveys",

issn = "0360-0300",

publisher = "Association for Computing Machinery (ACM)",

number = "2",

}

TY - JOUR

T1 - Bias in Reinforcement Learning

T2 - A Review in Healthcare Applications

AU - Smith, Benjamin

AU - Khojandi, Anahita

AU - Vasudevan, Rama

PY - 2023/9/15

Y1 - 2023/9/15

N2 - Reinforcement learning (RL) can assist in medical decision making using patient data collected in electronic health record (EHR) systems. RL, a type of machine learning, can use these data to develop treatment policies. However, RL models are typically trained using imperfect retrospective EHR data. Therefore, if care is not taken in training, RL policies can propagate existing bias in healthcare. Literature that considers and addresses the issues of bias and fairness in sequential decision making are reviewed. The major themes to mitigate bias that emerge relate to (1) data management; (2) algorithmic design; and (3) clinical understanding of the resulting policies.

AB - Reinforcement learning (RL) can assist in medical decision making using patient data collected in electronic health record (EHR) systems. RL, a type of machine learning, can use these data to develop treatment policies. However, RL models are typically trained using imperfect retrospective EHR data. Therefore, if care is not taken in training, RL policies can propagate existing bias in healthcare. Literature that considers and addresses the issues of bias and fairness in sequential decision making are reviewed. The major themes to mitigate bias that emerge relate to (1) data management; (2) algorithmic design; and (3) clinical understanding of the resulting policies.

KW - Reinforcement learning

KW - algorithmic bias

KW - bias management

KW - electronic health records

KW - treatment planning

UR - http://www.scopus.com/inward/record.url?scp=85174720069&partnerID=8YFLogxK

U2 - 10.1145/3609502

DO - 10.1145/3609502

M3 - Article

AN - SCOPUS:85174720069

SN - 0360-0300

VL - 56

JO - ACM Computing Surveys

JF - ACM Computing Surveys

IS - 2

M1 - 52

ER -

Bias in Reinforcement Learning: A Review in Healthcare Applications

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this