Triage Score: A Counterfactual Risk Assessment Instrument

Risk assessment instruments, or ``risk scores,'' are widely used in high-stakes decision-making settings such as medicine and the criminal justice system. A risk score predicts the likelihood of an undesired outcome if no intervention is made. Thus, a sufficiently high score is often interpreted as a recommendation to intervene. However, risk scores fail to account for what would happen if a decision-maker does intervene. This is problematic because effective decision-making requires consideration of how the intervention affects outcomes. We propose ``triage scores,'' which generalize risk scores using counterfactual utilities. Unlike risk scores, triage scores incorporate counterfactual outcomes under alternative decisions, enabling decision-makers to incorporate a wide range of ethical and practical factors. We illustrate the use of triage scores with an application to our own randomized controlled trial evaluating a pre-trial risk assessment instrument. Our analysis demonstrates that triage scores are able to capture richer utility structures than risk scores and yield substantively distinct results regarding policy evaluation and learning.