Methods for identifying patients at high risk for osteoporotic fractures, including dual-energy X-ray absorptiometry (DXA)1,2 and risk predictors like the Fracture Risk Assessment Tool (FRAX)3-6, are underutilized. We assessed the feasibility of automatic, opportunistic fracture risk evaluation based on routine abdomen or chest computed tomography (CT) scans. A CT-based predictor was created using three automatically generated bone imaging biomarkers (vertebral compression fractures (VCFs), simulated DXA T-scores and lumbar trabecular density) and CT metadata of age and sex. A cohort of 48,227 individuals (51.8% women) aged 50-90 with available CTs before 2012 (index date) were assessed for 5-year fracture risk using FRAX with no bone mineral density (BMD) input (FRAXnb) and the CT-based predictor. Predictions were compared to outcomes of major osteoporotic fractures and hip fractures during 2012-2017 (follow-up period). Compared with FRAXnb, the major osteoporotic fracture CT-based predictor presented better receiver operating characteristic area under curve (AUC), sensitivity and positive predictive value (PPV) (+1.9%, +2.4% and +0.7%, respectively). The AUC, sensitivity and PPV measures of the hip fracture CT-based predictor were noninferior to FRAXnb at a noninferiority margin of 1%. When FRAXnb inputs are not available, the initial evaluation of fracture risk can be done completely automatically based on a single abdomen or chest CT, which is often available for screening candidates7,8.

Methods for identifying patients at high risk for osteoporotic fractures, including dual-energy X-ray absorptiometry (DXA)1,2 and risk predictors like the Fracture Risk Assessment Tool (FRAX)3-6, are underutilized. We assessed the feasibility of automatic, opportunistic fracture risk evaluation based on routine abdomen or chest computed tomography (CT) scans. A CT-based predictor was created using three automatically generated bone imaging biomarkers (vertebral compression fractures (VCFs), simulated DXA T-scores and lumbar trabecular density) and CT metadata of age and sex. A cohort of 48,227 individuals (51.8% women) aged 50-90 with available CTs before 2012 (index date) were assessed for 5-year fracture risk using FRAX with no bone mineral density (BMD) input (FRAXnb) and the CT-based predictor. Predictions were compared to outcomes of major osteoporotic fractures and hip fractures during 2012-2017 (follow-up period). Compared with FRAXnb, the major osteoporotic fracture CT-based predictor presented better receiver operating characteristic area under curve (AUC), sensitivity and positive predictive value (PPV) (+1.9%, +2.4% and +0.7%, respectively). The AUC, sensitivity and PPV measures of the hip fracture CT-based predictor were noninferior to FRAXnb at a noninferiority margin of 1%. When FRAXnb inputs are not available, the initial evaluation of fracture risk can be done completely automatically based on a single abdomen or chest CT, which is often available for screening candidates7,8.

Currently, clinicians rely mostly on population-level treatment effects from RCTs, usually considering the treatment's benefits. This study proposes a process, focused on practical usability, for translating RCT data into personalized treatment recommendations that weighs benefits against harms and integrates subjective perceptions of relative severity. Intensive blood pressure treatment (IBPT) was selected as the test case to demonstrate the suggested process, which was divided into three phases: (1) Prediction models were developed using the Systolic Blood-Pressure Intervention Trial (SPRINT) data for benefits and adverse events of IBPT. The models were externally validated using retrospective Clalit Health Services (CHS) data; (2) Predicted risk reductions and increases from these models were used to create a yes/no IBPT recommendation by calculating a severity-weighted benefit-to-harm ratio; (3) Analysis outputs were summarized in a decision support tool. Based on the individual benefit-to-harm ratios, 62 and 84% of the SPRINT and CHS populations, respectively, would theoretically be recommended IBPT. The original SPRINT trial results of significant decrease in cardiovascular outcomes following IBPT persisted only in the group that received a "yes-treatment" recommendation by the suggested process, while the rate of serious adverse events was slightly higher in the "no-treatment" recommendation group. This process can be used to translate RCT data into individualized recommendations by identifying patients for whom the treatment's benefits outweigh the harms, while considering subjective views of perceived severity of the different outcomes. The proposed approach emphasizes clinical practicality by mimicking physicians' clinical decision-making process and integrating all recommendation outputs into a usable decision support tool.

OBJECTIVE: To illustrate the problem of subpopulation miscalibration, to adapt an algorithm for recalibration of the predictions, and to validate its performance.

MATERIALS AND METHODS: In this retrospective cohort study, we evaluated the calibration of predictions based on the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX) in the overall population and in subpopulations defined by the intersection of age, sex, ethnicity, socioeconomic status, and immigration history. We next applied the recalibration algorithm and assessed the change in calibration metrics, including calibration-in-the-large.

RESULTS: 1 021 041 patients were included in the PCE population, and 1 116 324 patients were included in the FRAX population. Baseline overall model calibration of the 2 tested models was good, but calibration in a substantial portion of the subpopulations was poor. After applying the algorithm, subpopulation calibration statistics were greatly improved, with the variance of the calibration-in-the-large values across all subpopulations reduced by 98.8% and 94.3% in the PCE and FRAX models, respectively.

DISCUSSION: Prediction models in medicine are increasingly common. Calibration, the agreement between predicted and observed risks, is commonly poor for subpopulations that were underrepresented in the development set of the models, resulting in bias and reduced performance for these subpopulations. In this work, we empirically evaluated an adapted version of the fairness algorithm designed by Hebert-Johnson et al. (2017) and demonstrated its use in improving subpopulation miscalibration.

CONCLUSION: A postprocessing and model-independent fairness algorithm for recalibration of predictive models greatly decreases the bias of subpopulation miscalibration and thus increases fairness and equality.

OBJECTIVE: To illustrate the problem of subpopulation miscalibration, to adapt an algorithm for recalibration of the predictions, and to validate its performance.

MATERIALS AND METHODS: In this retrospective cohort study, we evaluated the calibration of predictions based on the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX) in the overall population and in subpopulations defined by the intersection of age, sex, ethnicity, socioeconomic status, and immigration history. We next applied the recalibration algorithm and assessed the change in calibration metrics, including calibration-in-the-large.

RESULTS: 1 021 041 patients were included in the PCE population, and 1 116 324 patients were included in the FRAX population. Baseline overall model calibration of the 2 tested models was good, but calibration in a substantial portion of the subpopulations was poor. After applying the algorithm, subpopulation calibration statistics were greatly improved, with the variance of the calibration-in-the-large values across all subpopulations reduced by 98.8% and 94.3% in the PCE and FRAX models, respectively.

DISCUSSION: Prediction models in medicine are increasingly common. Calibration, the agreement between predicted and observed risks, is commonly poor for subpopulations that were underrepresented in the development set of the models, resulting in bias and reduced performance for these subpopulations. In this work, we empirically evaluated an adapted version of the fairness algorithm designed by Hebert-Johnson et al. (2017) and demonstrated its use in improving subpopulation miscalibration.

CONCLUSION: A postprocessing and model-independent fairness algorithm for recalibration of predictive models greatly decreases the bias of subpopulation miscalibration and thus increases fairness and equality.

At the COVID-19 pandemic onset, when individual-level data of COVID-19 patients were not yet available, there was already a need for risk predictors to support prevention and treatment decisions. Here, we report a hybrid strategy to create such a predictor, combining the development of a baseline severe respiratory infection risk predictor and a post-processing method to calibrate the predictions to reported COVID-19 case-fatality rates. With the accumulation of a COVID-19 patient cohort, this predictor is validated to have good discrimination (area under the receiver-operating characteristics curve of 0.943) and calibration (markedly improved compared to that of the baseline predictor). At a 5% risk threshold, 15% of patients are marked as high-risk, achieving a sensitivity of 88%. We thus demonstrate that even at the onset of a pandemic, shrouded in epidemiologic fog of war, it is possible to provide a useful risk predictor, now widely used in a large healthcare organization.

At the COVID-19 pandemic onset, when individual-level data of COVID-19 patients were not yet available, there was already a need for risk predictors to support prevention and treatment decisions. Here, we report a hybrid strategy to create such a predictor, combining the development of a baseline severe respiratory infection risk predictor and a post-processing method to calibrate the predictions to reported COVID-19 case-fatality rates. With the accumulation of a COVID-19 patient cohort, this predictor is validated to have good discrimination (area under the receiver-operating characteristics curve of 0.943) and calibration (markedly improved compared to that of the baseline predictor). At a 5% risk threshold, 15% of patients are marked as high-risk, achieving a sensitivity of 88%. We thus demonstrate that even at the onset of a pandemic, shrouded in epidemiologic fog of war, it is possible to provide a useful risk predictor, now widely used in a large healthcare organization.