Critical Appraisal for Racial and Ethnic Equity in Clinical Prediction Models extension: Development of a critical appraisal tool extension to assess racial and ethnic equity-related risk of bias for clinical prediction models

INTRODUCTION: Despite mounting evidence that the inclusion of race and ethnicity in clinical prediction models may contribute to health disparities, existing critical appraisal tools do not directly address such equity considerations.

OBJECTIVE: This study developed a critical appraisal tool extension to assess algorithmic bias in clinical prediction models.

METHODS: A modified e-Delphi approach was utilized to develop and obtain expert consensus on a set of racial and ethnic equity-based signaling questions for appraisal of risk of bias in clinical prediction models. Through a series of virtual meetings, initial pilot application, and an online survey, individuals with expertise in clinical prediction model development, systematic review methodology, and health equity developed and refined this tool.

RESULTS: Consensus was reached for ten equity-based signaling questions, which led to the development of the Critical Appraisal for Racial and Ethnic Equity in Clinical Prediction Models (CARE-CPM) extension. This extension is intended for use along with existing critical appraisal tools for clinical prediction models.

CONCLUSION: CARE-CPM provides a valuable risk-of-bias assessment tool extension for clinical prediction models to identify potential algorithmic bias and health equity concerns. Further research is needed to test usability, interrater reliability, and application to decision-makers.

Agency for Healthcare Research and Quality

Shazia M Siddique, Corinne V Evans, Michael Harhay, Eric S Johnson, Jaya Aysola, Gary E Weissman, Nikhil K Mull, Emilia Flores, Harald Schmidt, Kelley Tipton, Brian Leas, Jennifer S Lin