Due to increasing concern that the use of prediction models in healthcare exacerbate existing disparities, the current study evaluated racial/ethnic differences in the performance of statistical models that predict suicide. The prediction models used data from outpatient mental health visits (January 1, 2009 to December 31, 2017) to seven large integrated healthcare systems by patients 13 years or older. Suicide death in the 90 days after a mental health visit was predicted from data collected from 13,980,570 visits by 1,433,543 patients (65% female, mean age 42 [18] years). The area-under-the curve (AUC), which measures model discrimination, was high at the population level and varied considerably by racial/ethnic group. AUCs were highest for visits from White, Hispanic, and Asian patients and lowest for visits from Black and American Indian/Alaskan Native patients and those with unrecorded race/ethnicity. Similar trends were observed for sensitivity (equivalent to the true positive rate) of the suicide predictions models. Defining race/ethnicity-specific thresholds improved sensitivity of models inconsistently for subgroups, in some cases increasing, but in other cases, decreasing sensitivity. Clinical implementation of these models to prevent suicide would likely provide less benefit to Black and American Indian/Alaskan Native patients and patients without a recorded race/ethnicity because fewer suicides would be identified. Increasing sensitivity within racial/ethnic groups by defining separate thresholds for intervention may also be harmful in some cases by exposing low risk individuals in these populations to unnecessary and potentially intrusive interventions. To realize the full benefits of prediction models in clinical practice, they must be assessed in the patient populations in whom they will be used, including vulnerable groups in that population.
Reference:
Coley RY et al. JAMA Psychiatry 2021; epub ahead of print. Abstract
Additional Information: