Heart failure (HF) prevalence is increasing worldwide and is associated with significant morbidity and mortality. Guidelines emphasize prevention in those at-risk, but HF-specific risk prediction equations developed in United States population-based cohorts lack external validation in large, real-world datasets outside of the United States. The purpose of this study was to assess the model performance of the pooled cohort equations to prevent HF (PCP-HF) within a contemporary electronic health record for 5- and 10-year risk. Using a retrospective cohort study design of Israeli residents between 2008 and 2018 with continuous membership until end of follow-up, HF, or death, we quantified 5- and 10-year estimated risks of HF using the PCP-HF equations, which integrate demographics (age, gender, and race) and risk factors (body mass index, systolic blood pressure, glucose, medication use for hypertension or diabetes, and smoking status). Of 1,394,411 patients included, 56% were women with mean age of 49.6 (SD 13.2) years. Incident HF occurred in 1.2% and 4.5% of participants over 5 and 10 years of follow-up. The PCP-HF model had excellent discrimination for 5- and 10-year predictions of incident HF (C Statistic 0.82 [0.82 to 0.82] and 0.84 [0.84 to 0.84]), respectively. In conclusion, HF-specific risk equations (PCP-HF) accurately predict the risk of incident HF in ambulatory and hospitalized patients using routinely available clinical data.

Heart failure (HF) prevalence is increasing worldwide and is associated with significant morbidity and mortality. Guidelines emphasize prevention in those at-risk, but HF-specific risk prediction equations developed in United States population-based cohorts lack external validation in large, real-world datasets outside of the United States. The purpose of this study was to assess the model performance of the pooled cohort equations to prevent HF (PCP-HF) within a contemporary electronic health record for 5- and 10-year risk. Using a retrospective cohort study design of Israeli residents between 2008 and 2018 with continuous membership until end of follow-up, HF, or death, we quantified 5- and 10-year estimated risks of HF using the PCP-HF equations, which integrate demographics (age, gender, and race) and risk factors (body mass index, systolic blood pressure, glucose, medication use for hypertension or diabetes, and smoking status). Of 1,394,411 patients included, 56% were women with mean age of 49.6 (SD 13.2) years. Incident HF occurred in 1.2% and 4.5% of participants over 5 and 10 years of follow-up. The PCP-HF model had excellent discrimination for 5- and 10-year predictions of incident HF (C Statistic 0.82 [0.82 to 0.82] and 0.84 [0.84 to 0.84]), respectively. In conclusion, HF-specific risk equations (PCP-HF) accurately predict the risk of incident HF in ambulatory and hospitalized patients using routinely available clinical data.

OBJECTIVE: To illustrate the problem of subpopulation miscalibration, to adapt an algorithm for recalibration of the predictions, and to validate its performance.

MATERIALS AND METHODS: In this retrospective cohort study, we evaluated the calibration of predictions based on the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX) in the overall population and in subpopulations defined by the intersection of age, sex, ethnicity, socioeconomic status, and immigration history. We next applied the recalibration algorithm and assessed the change in calibration metrics, including calibration-in-the-large.

RESULTS: 1 021 041 patients were included in the PCE population, and 1 116 324 patients were included in the FRAX population. Baseline overall model calibration of the 2 tested models was good, but calibration in a substantial portion of the subpopulations was poor. After applying the algorithm, subpopulation calibration statistics were greatly improved, with the variance of the calibration-in-the-large values across all subpopulations reduced by 98.8% and 94.3% in the PCE and FRAX models, respectively.

DISCUSSION: Prediction models in medicine are increasingly common. Calibration, the agreement between predicted and observed risks, is commonly poor for subpopulations that were underrepresented in the development set of the models, resulting in bias and reduced performance for these subpopulations. In this work, we empirically evaluated an adapted version of the fairness algorithm designed by Hebert-Johnson et al. (2017) and demonstrated its use in improving subpopulation miscalibration.

CONCLUSION: A postprocessing and model-independent fairness algorithm for recalibration of predictive models greatly decreases the bias of subpopulation miscalibration and thus increases fairness and equality.

OBJECTIVE: To illustrate the problem of subpopulation miscalibration, to adapt an algorithm for recalibration of the predictions, and to validate its performance.

MATERIALS AND METHODS: In this retrospective cohort study, we evaluated the calibration of predictions based on the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX) in the overall population and in subpopulations defined by the intersection of age, sex, ethnicity, socioeconomic status, and immigration history. We next applied the recalibration algorithm and assessed the change in calibration metrics, including calibration-in-the-large.

RESULTS: 1 021 041 patients were included in the PCE population, and 1 116 324 patients were included in the FRAX population. Baseline overall model calibration of the 2 tested models was good, but calibration in a substantial portion of the subpopulations was poor. After applying the algorithm, subpopulation calibration statistics were greatly improved, with the variance of the calibration-in-the-large values across all subpopulations reduced by 98.8% and 94.3% in the PCE and FRAX models, respectively.

DISCUSSION: Prediction models in medicine are increasingly common. Calibration, the agreement between predicted and observed risks, is commonly poor for subpopulations that were underrepresented in the development set of the models, resulting in bias and reduced performance for these subpopulations. In this work, we empirically evaluated an adapted version of the fairness algorithm designed by Hebert-Johnson et al. (2017) and demonstrated its use in improving subpopulation miscalibration.

CONCLUSION: A postprocessing and model-independent fairness algorithm for recalibration of predictive models greatly decreases the bias of subpopulation miscalibration and thus increases fairness and equality.