TY - JOUR
T1 - A cross-study analysis of drug response prediction in cancer cell lines
AU - Xia, Fangfang
AU - Allen, Jonathan
AU - Balaprakash, Prasanna
AU - Brettin, Thomas
AU - Garcia-Cardona, Cristina
AU - Clyde, Austin
AU - Cohn, Judith
AU - Doroshow, James
AU - Duan, Xiaotian
AU - Dubinkina, Veronika
AU - Evrard, Yvonne
AU - Fan, Ya Ju
AU - Gans, Jason
AU - He, Stewart
AU - Lu, Pinyi
AU - Maslov, Sergei
AU - Partin, Alexander
AU - Shukla, Maulik
AU - Stahlberg, Eric
AU - Wozniak, Justin M.
AU - Yoo, Hyunseung
AU - Zaki, George
AU - Zhu, Yitan
AU - Stevens, Rick
N1 - Publisher Copyright:
© 2021 The Author(s) 2021. Published by Oxford University Press.
PY - 2022/1/1
Y1 - 2022/1/1
N2 - To enable personalized cancer treatment, machine learning models have been developed to predict drug response as a function of tumor and drug features. However, most algorithm development efforts have relied on cross-validation within a single study to assess model accuracy. While an essential first step, cross-validation within a biological data set typically provides an overly optimistic estimate of the prediction performance on independent test sets. To provide a more rigorous assessment of model generalizability between different studies, we use machine learning to analyze five publicly available cell line-based data sets: National Cancer Institute 60, ancer Therapeutics Response Portal (CTRP), Genomics of Drug Sensitivity in Cancer, Cancer Cell Line Encyclopedia and Genentech Cell Line Screening Initiative (gCSI). Based on observed experimental variability across studies, we explore estimates of prediction upper bounds. We report performance results of a variety of machine learning models, with a multitasking deep neural network achieving the best cross-study generalizability. By multiple measures, models trained on CTRP yield the most accurate predictions on the remaining testing data, and gCSI is the most predictable among the cell line data sets included in this study. With these experiments and further simulations on partial data, two lessons emerge: (1) differences in viability assays can limit model generalizability across studies and (2) drug diversity, more than tumor diversity, is crucial for raising model generalizability in preclinical screening.
AB - To enable personalized cancer treatment, machine learning models have been developed to predict drug response as a function of tumor and drug features. However, most algorithm development efforts have relied on cross-validation within a single study to assess model accuracy. While an essential first step, cross-validation within a biological data set typically provides an overly optimistic estimate of the prediction performance on independent test sets. To provide a more rigorous assessment of model generalizability between different studies, we use machine learning to analyze five publicly available cell line-based data sets: National Cancer Institute 60, ancer Therapeutics Response Portal (CTRP), Genomics of Drug Sensitivity in Cancer, Cancer Cell Line Encyclopedia and Genentech Cell Line Screening Initiative (gCSI). Based on observed experimental variability across studies, we explore estimates of prediction upper bounds. We report performance results of a variety of machine learning models, with a multitasking deep neural network achieving the best cross-study generalizability. By multiple measures, models trained on CTRP yield the most accurate predictions on the remaining testing data, and gCSI is the most predictable among the cell line data sets included in this study. With these experiments and further simulations on partial data, two lessons emerge: (1) differences in viability assays can limit model generalizability across studies and (2) drug diversity, more than tumor diversity, is crucial for raising model generalizability in preclinical screening.
KW - deep learning
KW - drug response prediction
KW - drug sensitivity
KW - precision oncology
UR - http://www.scopus.com/inward/record.url?scp=85123814526&partnerID=8YFLogxK
U2 - 10.1093/bib/bbab356
DO - 10.1093/bib/bbab356
M3 - Article
C2 - 34524425
AN - SCOPUS:85123814526
SN - 1467-5463
VL - 23
JO - Briefings in bioinformatics
JF - Briefings in bioinformatics
IS - 1
M1 - bbab356
ER -