High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

Sayera Dhaubhadel; Kumkum Ganguly; Ruy M. Ribeiro; Judith D. Cohn; James M. Hyman; Nicolas W. Hengartner; Beauty Kolade; Anna Singley; Tanmoy Bhattacharya; Patrick Finley; Drew Levin; Haedi Thelen; Kelly Cho; Lauren Costa; Yuk Lam Ho; Amy C. Justice; John Pestian; Daniel Santel; Rafael Zamora-Resendiz; Silvia Crivelli; Suzanne Tamang; Susana Martins; Jodie Trafton; David W. Oslin; Jean C. Beckham; Nathan A. Kimbrel; Khushbu Agarwal; Allison E. Ashley-Koch; Mihaela Aslan; Edmond Begoli; Ben Brown; Patrick S. Calhoun; Kei Hoi Cheung; Sutanay Choudhury; Ashley M. Cliff; Leticia Cuellar-Hengartner; Haedi E. Deangelis; Michelle F. Dennis; Patrick D. Finley; Michael R. Garvin; Joel E. Gelernter; Lauren P. Hair; Colby Ham; Phillip D. Harvey; Elizabeth R. Hauser; Michael A. Hauser; Nick W. Hengartner; Daniel A. Jacobson; Jessica Jones; Piet C. Jones; David Kainer; Alan D. Kaplan; Ira R. Katz; Rachel L. Kember; Angela C. Kirby; John C. Ko; John Lagergren; Matthew Lane; Daniel F. Levey; Jennifer H. Lindquist; Xianlian Liu; Ravi K. Madduri; Carrie Manore; Carianne Martinez; John F. McCarthy; Mikaela Mc Devitt Cashman; J. Izaak Miller; Destinee Morrow; Mirko Pavicic-Venegas; Saiju Pyarajan; Xue J. Qin; Nallakkandi Rajeevan; Christine M. Ramsey; Ruy Ribeiro; Alex Rodriguez; Jonathon Romero; Yunling Shi; Murray B. Stein; Kyle A. Sullivan; Ning Sun; Suzanne R. Tamang; Alice Townsend; Jodie A. Trafton; Angelica Walker; Xiange Wang; Victoria Wangia-Anderson; Renji Yang; Shinjae Yoo; Hongyu Zhao; Benjamin H. McMahon

doi:10.1038/s41598-024-51762-9

High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

Sayera Dhaubhadel, Kumkum Ganguly, Ruy M. Ribeiro, Judith D. Cohn, James M. Hyman, Nicolas W. Hengartner, Beauty Kolade, Anna Singley, Tanmoy Bhattacharya, Patrick Finley, Drew Levin, Haedi Thelen, Kelly Cho, Lauren Costa, Yuk Lam Ho, Amy C. Justice, John Pestian, Daniel Santel, Rafael Zamora-Resendiz, Silvia CrivelliSuzanne Tamang, Susana Martins, Jodie Trafton, David W. Oslin, Jean C. Beckham, Nathan A. Kimbrel, Khushbu Agarwal, Allison E. Ashley-Koch, Mihaela Aslan, Edmond Begoli, Ben Brown, Patrick S. Calhoun, Kei Hoi Cheung, Sutanay Choudhury, Ashley M. Cliff, Leticia Cuellar-Hengartner, Haedi E. Deangelis, Michelle F. Dennis, Patrick D. Finley, Michael R. Garvin, Joel E. Gelernter, Lauren P. Hair, Colby Ham, Phillip D. Harvey, Elizabeth R. Hauser, Michael A. Hauser, Nick W. Hengartner, Daniel A. Jacobson, Jessica Jones, Piet C. Jones, David Kainer, Alan D. Kaplan, Ira R. Katz, Rachel L. Kember, Angela C. Kirby, John C. Ko, John Lagergren, Matthew Lane, Daniel F. Levey, Jennifer H. Lindquist, Xianlian Liu, Ravi K. Madduri, Carrie Manore, Carianne Martinez, John F. McCarthy, Mikaela Mc Devitt Cashman, J. Izaak Miller, Destinee Morrow, Mirko Pavicic-Venegas, Saiju Pyarajan, Xue J. Qin, Nallakkandi Rajeevan, Christine M. Ramsey, Ruy Ribeiro, Alex Rodriguez, Jonathon Romero, Yunling Shi, Murray B. Stein, Kyle A. Sullivan, Ning Sun, Suzanne R. Tamang, Alice Townsend, Jodie A. Trafton, Angelica Walker, Xiange Wang, Victoria Wangia-Anderson, Renji Yang, Shinjae Yoo, Hongyu Zhao, Benjamin H. McMahon

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of ∼ 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.

Original language	English
Article number	1793
Journal	Scientific Reports
Volume	14
Issue number	1
DOIs	https://doi.org/10.1038/s41598-024-51762-9
State	Published - Dec 2024

Funding

This research is based on data from the Million Veteran Program, Office of Research and Development, Veterans Health Administration, supported by award #MVP011. This publication does not represent the views of the Department of Veteran Affairs or the United States Government. J.C. Beckham was also supported by a Senior Research Career Scientist Award (#lK6BX003777) from CSR &D. We would like to thank Eric Caine for helpful discussions during problem formulation and Ethan Romero–Severson for helpful critiques on multiple drafts of this manuscript. This manuscript has been authored by Triad National Security, LLC under Contract No. 89233218CNA000001 with the U.S. Department of Energy/National Nuclear Security Administration. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript or allow others to do so, for United States Government purposes. The Government has also granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in the code and data, within this manuscript, to reproduce, prepare derivative works, and perform publicly and display publicly, by or on behalf of the Government. NEITHER THE GOVERNMENT NOR THE CONTRACTOR MAKES ANY WARRANTY, EXPRESS OR IMPLIED, OR ASSUMES ANY LIABILITY FOR THE USE OF CODE WITHIN THIS MANUSCRIPT. This notice including this sentence must appear on any copies of the code. This research is based on data from the Million Veteran Program, Office of Research and Development, Veterans Health Administration, supported by award #MVP011. This publication does not represent the views of the Department of Veteran Affairs or the United States Government. J.C. Beckham was also supported by a Senior Research Career Scientist Award (#lK6BX003777) from CSR &D. We would like to thank Eric Caine for helpful discussions during problem formulation and Ethan Romero–Severson for helpful critiques on multiple drafts of this manuscript. This manuscript has been authored by Triad National Security, LLC under Contract No. 89233218CNA000001 with the U.S. Department of Energy/National Nuclear Security Administration. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript or allow others to do so, for United States Government purposes. The Government has also granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in the code and data, within this manuscript, to reproduce, prepare derivative works, and perform publicly and display publicly, by or on behalf of the Government. NEITHER THE GOVERNMENT NOR THE CONTRACTOR MAKES ANY WARRANTY, EXPRESS OR IMPLIED, OR ASSUMES ANY LIABILITY FOR THE USE OF CODE WITHIN THIS MANUSCRIPT. This notice including this sentence must appear on any copies of the code.

Funders	Funder number
Million Veteran Program
United States Government
Center for Scientific Review	89233218CNA000001
Center for Scientific Review
National Nuclear Security Administration
Office of Research and Development
Health Services Research and Development	6BX003777
Health Services Research and Development

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1038/s41598-024-51762-9

Cite this

Dhaubhadel, S., Ganguly, K., Ribeiro, R. M., Cohn, J. D., Hyman, J. M., Hengartner, N. W., Kolade, B., Singley, A., Bhattacharya, T., Finley, P., Levin, D., Thelen, H., Cho, K., Costa, L., Ho, Y. L., Justice, A. C., Pestian, J., Santel, D., Zamora-Resendiz, R., ... McMahon, B. H. (2024). High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning. Scientific Reports, 14(1), Article 1793. https://doi.org/10.1038/s41598-024-51762-9

@article{d914d90daccf41f8908ced7bdf3fb862,

title = "High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning",

abstract = "We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of ∼ 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.",

author = "Sayera Dhaubhadel and Kumkum Ganguly and Ribeiro, {Ruy M.} and Cohn, {Judith D.} and Hyman, {James M.} and Hengartner, {Nicolas W.} and Beauty Kolade and Anna Singley and Tanmoy Bhattacharya and Patrick Finley and Drew Levin and Haedi Thelen and Kelly Cho and Lauren Costa and Ho, {Yuk Lam} and Justice, {Amy C.} and John Pestian and Daniel Santel and Rafael Zamora-Resendiz and Silvia Crivelli and Suzanne Tamang and Susana Martins and Jodie Trafton and Oslin, {David W.} and Beckham, {Jean C.} and Kimbrel, {Nathan A.} and Khushbu Agarwal and Ashley-Koch, {Allison E.} and Mihaela Aslan and Edmond Begoli and Ben Brown and Calhoun, {Patrick S.} and Cheung, {Kei Hoi} and Sutanay Choudhury and Cliff, {Ashley M.} and Leticia Cuellar-Hengartner and Deangelis, {Haedi E.} and Dennis, {Michelle F.} and Finley, {Patrick D.} and Garvin, {Michael R.} and Gelernter, {Joel E.} and Hair, {Lauren P.} and Colby Ham and Harvey, {Phillip D.} and Hauser, {Elizabeth R.} and Hauser, {Michael A.} and Hengartner, {Nick W.} and Jacobson, {Daniel A.} and Jessica Jones and Jones, {Piet C.} and David Kainer and Kaplan, {Alan D.} and Katz, {Ira R.} and Kember, {Rachel L.} and Kirby, {Angela C.} and Ko, {John C.} and John Lagergren and Matthew Lane and Levey, {Daniel F.} and Lindquist, {Jennifer H.} and Xianlian Liu and Madduri, {Ravi K.} and Carrie Manore and Carianne Martinez and McCarthy, {John F.} and Cashman, {Mikaela Mc Devitt} and Miller, {J. Izaak} and Destinee Morrow and Mirko Pavicic-Venegas and Saiju Pyarajan and Qin, {Xue J.} and Nallakkandi Rajeevan and Ramsey, {Christine M.} and Ruy Ribeiro and Alex Rodriguez and Jonathon Romero and Yunling Shi and Stein, {Murray B.} and Sullivan, {Kyle A.} and Ning Sun and Tamang, {Suzanne R.} and Alice Townsend and Trafton, {Jodie A.} and Angelica Walker and Xiange Wang and Victoria Wangia-Anderson and Renji Yang and Shinjae Yoo and Hongyu Zhao and McMahon, {Benjamin H.}",

note = "Publisher Copyright: {\textcopyright} 2024, This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply.",

year = "2024",

month = dec,

doi = "10.1038/s41598-024-51762-9",

language = "English",

volume = "14",

journal = "Scientific Reports",

issn = "2045-2322",

publisher = "Nature Research",

number = "1",

}

Dhaubhadel, S, Ganguly, K, Ribeiro, RM, Cohn, JD, Hyman, JM, Hengartner, NW, Kolade, B, Singley, A, Bhattacharya, T, Finley, P, Levin, D, Thelen, H, Cho, K, Costa, L, Ho, YL, Justice, AC, Pestian, J, Santel, D, Zamora-Resendiz, R, Crivelli, S, Tamang, S, Martins, S, Trafton, J, Oslin, DW, Beckham, JC, Kimbrel, NA, Agarwal, K, Ashley-Koch, AE, Aslan, M, Begoli, E, Brown, B, Calhoun, PS, Cheung, KH, Choudhury, S, Cliff, AM, Cuellar-Hengartner, L, Deangelis, HE, Dennis, MF, Finley, PD, Garvin, MR, Gelernter, JE, Hair, LP, Ham, C, Harvey, PD, Hauser, ER, Hauser, MA, Hengartner, NW, Jacobson, DA, Jones, J, Jones, PC, Kainer, D, Kaplan, AD, Katz, IR, Kember, RL, Kirby, AC, Ko, JC, Lagergren, J, Lane, M, Levey, DF, Lindquist, JH, Liu, X, Madduri, RK, Manore, C, Martinez, C, McCarthy, JF, Cashman, MMD, Miller, JI, Morrow, D, Pavicic-Venegas, M, Pyarajan, S, Qin, XJ, Rajeevan, N, Ramsey, CM, Ribeiro, R, Rodriguez, A, Romero, J, Shi, Y, Stein, MB, Sullivan, KA, Sun, N, Tamang, SR, Townsend, A, Trafton, JA, Walker, A, Wang, X, Wangia-Anderson, V, Yang, R, Yoo, S, Zhao, H & McMahon, BH 2024, 'High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning', Scientific Reports, vol. 14, no. 1, 1793. https://doi.org/10.1038/s41598-024-51762-9

TY - JOUR

T1 - High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

AU - Dhaubhadel, Sayera

AU - Ganguly, Kumkum

AU - Ribeiro, Ruy M.

AU - Cohn, Judith D.

AU - Hyman, James M.

AU - Hengartner, Nicolas W.

AU - Kolade, Beauty

AU - Singley, Anna

AU - Bhattacharya, Tanmoy

AU - Finley, Patrick

AU - Levin, Drew

AU - Thelen, Haedi

AU - Cho, Kelly

AU - Costa, Lauren

AU - Ho, Yuk Lam

AU - Justice, Amy C.

AU - Pestian, John

AU - Santel, Daniel

AU - Zamora-Resendiz, Rafael

AU - Crivelli, Silvia

AU - Tamang, Suzanne

AU - Martins, Susana

AU - Trafton, Jodie

AU - Oslin, David W.

AU - Beckham, Jean C.

AU - Kimbrel, Nathan A.

AU - Agarwal, Khushbu

AU - Ashley-Koch, Allison E.

AU - Aslan, Mihaela

AU - Begoli, Edmond

AU - Brown, Ben

AU - Calhoun, Patrick S.

AU - Cheung, Kei Hoi

AU - Choudhury, Sutanay

AU - Cliff, Ashley M.

AU - Cuellar-Hengartner, Leticia

AU - Deangelis, Haedi E.

AU - Dennis, Michelle F.

AU - Finley, Patrick D.

AU - Garvin, Michael R.

AU - Gelernter, Joel E.

AU - Hair, Lauren P.

AU - Ham, Colby

AU - Harvey, Phillip D.

AU - Hauser, Elizabeth R.

AU - Hauser, Michael A.

AU - Hengartner, Nick W.

AU - Jacobson, Daniel A.

AU - Jones, Jessica

AU - Jones, Piet C.

AU - Kainer, David

AU - Kaplan, Alan D.

AU - Katz, Ira R.

AU - Kember, Rachel L.

AU - Kirby, Angela C.

AU - Ko, John C.

AU - Lagergren, John

AU - Lane, Matthew

AU - Levey, Daniel F.

AU - Lindquist, Jennifer H.

AU - Liu, Xianlian

AU - Madduri, Ravi K.

AU - Manore, Carrie

AU - Martinez, Carianne

AU - McCarthy, John F.

AU - Cashman, Mikaela Mc Devitt

AU - Miller, J. Izaak

AU - Morrow, Destinee

AU - Pavicic-Venegas, Mirko

AU - Pyarajan, Saiju

AU - Qin, Xue J.

AU - Rajeevan, Nallakkandi

AU - Ramsey, Christine M.

AU - Ribeiro, Ruy

AU - Rodriguez, Alex

AU - Romero, Jonathon

AU - Shi, Yunling

AU - Stein, Murray B.

AU - Sullivan, Kyle A.

AU - Sun, Ning

AU - Tamang, Suzanne R.

AU - Townsend, Alice

AU - Trafton, Jodie A.

AU - Walker, Angelica

AU - Wang, Xiange

AU - Wangia-Anderson, Victoria

AU - Yang, Renji

AU - Yoo, Shinjae

AU - Zhao, Hongyu

AU - McMahon, Benjamin H.

PY - 2024/12

Y1 - 2024/12

N2 - We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of ∼ 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.

AB - We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of ∼ 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.

UR - http://www.scopus.com/inward/record.url?scp=85182812331&partnerID=8YFLogxK

U2 - 10.1038/s41598-024-51762-9

DO - 10.1038/s41598-024-51762-9

M3 - Article

C2 - 38245528

AN - SCOPUS:85182812331

SN - 2045-2322

VL - 14

JO - Scientific Reports

JF - Scientific Reports

IS - 1

M1 - 1793

ER -

High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

Abstract

Funding

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this