Prognostic Models for Stable Coronary Artery Disease
Prognostic Models for Stable Coronary Artery Disease
In a population-based study of >100 000 stable patients with a range of previous CAD phenotypes, we found that prognostic models combining a wide range of clinical data commonly available prior to the decision for further investigation can identify patients at high risk of long-term mortality and coronary events. We demonstrate how real-world clinical data (as distinct from research data) contribute important prognostic information in unselected patients. Unlike previous reports of prognostic models, we focus on potential clinical usefulness and demonstrate how each predictor usefully improves predictions beyond more simple models. Importantly, we confirm that the models have good calibration and discrimination when applied to an external study. These validated prognostic models could be used clinically to support risk assessment according to clinical variables, which is 'essential for determining optimal treatment strategies'.
Current practice guidelines informed several aspects of this analysis. Because risks in SCAD are heterogeneous and many patients have low risk, the 2012 ACCF/AHA and 2006 ESC guidelines propose a stepwise risk assessment to inform management decisions. The assessment should start with evaluating all patients based on simple clinical parameters, followed by advanced investigations (imaging and stress testing) in selected patients, followed by angiography conditional on results from earlier steps. The CALIBER prognostic models incorporate the broad range of clinical characteristics highlighted by these guidelines for the initial evaluation step, many of which (deprivation, atrial fibrillation, cancer, liver disease, depression, anxiety, and haemoglobin) have not previously been incorporated in prognostic models for SCAD. Importantly, to make the evaluation of the models relevant to the guidelines, we used the ACCF/AHA definition of low, intermediate, and high risk to estimate the NRI and life years gained from applying these models. Hence, our evaluation outcomes are clinically relevant.
We present both all-cause mortality and non-fatal MI or coronary death as outcomes because these risks differ and the best course of action in terms of risk management and priorities requires evaluation of both mortality and coronary morbidity. Some prognostic factors included in the CALIBER models were much more useful for predicting all-cause mortality than coronary risks and vice versa. Hence, although smoking, diabetes, CVD comorbidities, and biomarkers were good predictors of either outcome, non-CVD comorbidities, depression, and anxiety were mainly useful for predicting non-coronary death. Importantly, the contribution of hypertension to model discrimination was marginal and confined to coronary events. Since blood pressure lowering trials among people with SCAD demonstrate a reduction in both coronary and total mortality rates, the prognostic effect of hypertension may be obscured by treatment.
Widely reported measures of model performance, such as the C-index and the NRI, do not provide clinicians with readily interpretable evidence for making effective, and cost-effective decisions about which patients should be further investigated. Following recommendations, we therefore sought to estimate the potential impact on patient outcomes of using the CALIBER prognostic models for clinical decision making. Using these models to identify patients at high risk (defined by guidelines as 3% annual mortality) and offer a management strategy with a hazard ratio 0.8 (equivalent to ~20% absolute risk reduction) would save an additional 13–16 life years or 15–18 coronary-event-free life years over a 5-year time horizon compared with a model including just age, sex, and social deprivation. Hence, since the models are based on clinically available data (i.e. there is no cost increment of collecting new data), our analysis demonstrates that screening with the models is likely to be cost-effective. However, economic evaluation would be needed to establish their appropriateness in practice.
Stringent external validation is essential to ensure that a prognostic model would be applicable to the rest of the population than the patients in our cohort. However, a model can achieve impressive performance (as high as in the development data) if applied to an external data set that originates from a similar source of data (e.g. validation of QRISK in data from a similar GP database system). To make our external validation rigorous, we therefore used a data set with major differences from the development data set; the data were collected differently (manually abstracted from case records), the patients were at higher risk (received angiography due to chest pain), and the study period was about a decade earlier with markedly different background use of risk lowering medication. Hence, it was reassuring that the models performed well in this external setting. However, as with any prognostic model, there is an ongoing need for further validation in external data sets and recalibration for different populations and time periods.
We propose that the CALIBER prognostic models are implemented in electronic health records in clinical practice. This should be done alongside evaluation of the impact on decisions and, ultimately, patient outcomes. The models may also be used to support adherence to existing therapies, guide frequency of follow-up, plan trials of new therapies, and serve as a reference on which to evaluate the extent to which novel biomarkers might usefully further stratify risk assessment.
A limitation in our study is that the electronic health record does not capture all the information available to clinicians prior to imaging. Thus, while clinicians assess symptom severity, this is not recorded in standard fashion. However, we found that a proxy for symptom severity, the use of long-acting nitrates, or repeat prescription for short nitrates did add prognostic value. Although we had information from the resting ECG on heart rate and the presence of atrial fibrillation, resting ST segment, T wave, or other changes were seldom coded. A further limitation was that for external evaluation, we ignored (assumed absent) comorbidities not collected in the external data set. However, the missing predictors had low impact on the performance of the models in the development data; thus, we do not expect this to have affected the evaluation. Further, in order to represent all relevant clinical parameters that are commonly considered in clinical assessment, the models included a large number of variables. However, these variables are routinely recorded in electronic health records, so their collection is not associated with extra costs. Also, the large size of the data set used to develop the models reduces the likelihood of overfitting. Finally, only multiplicative models were considered; models fitted in the additive scale are a plausible alternative that was not explored.
In summary, we present validated prognostic models for estimating risk of all-cause mortality and coronary outcomes based on clinical parameters that are commonly available in all people with stable coronary disease. These models can be implemented alongside further medical investigations to support medical decision making. However, as with any new prognostic model, further independent evaluation is required in different settings including different electronic health record systems, health care organizations, and geographic locations to guide use in clinical practice. A risk calculator is available online (www.caliberresearch.org/model).
We registered the research protocol at clinical trials.gov (NCT01609465) and received approval from the Independent Scientific Advisory Committee (10_160) and the MINAP Academic Group (11-CBR-3).
Discussion
In a population-based study of >100 000 stable patients with a range of previous CAD phenotypes, we found that prognostic models combining a wide range of clinical data commonly available prior to the decision for further investigation can identify patients at high risk of long-term mortality and coronary events. We demonstrate how real-world clinical data (as distinct from research data) contribute important prognostic information in unselected patients. Unlike previous reports of prognostic models, we focus on potential clinical usefulness and demonstrate how each predictor usefully improves predictions beyond more simple models. Importantly, we confirm that the models have good calibration and discrimination when applied to an external study. These validated prognostic models could be used clinically to support risk assessment according to clinical variables, which is 'essential for determining optimal treatment strategies'.
Current practice guidelines informed several aspects of this analysis. Because risks in SCAD are heterogeneous and many patients have low risk, the 2012 ACCF/AHA and 2006 ESC guidelines propose a stepwise risk assessment to inform management decisions. The assessment should start with evaluating all patients based on simple clinical parameters, followed by advanced investigations (imaging and stress testing) in selected patients, followed by angiography conditional on results from earlier steps. The CALIBER prognostic models incorporate the broad range of clinical characteristics highlighted by these guidelines for the initial evaluation step, many of which (deprivation, atrial fibrillation, cancer, liver disease, depression, anxiety, and haemoglobin) have not previously been incorporated in prognostic models for SCAD. Importantly, to make the evaluation of the models relevant to the guidelines, we used the ACCF/AHA definition of low, intermediate, and high risk to estimate the NRI and life years gained from applying these models. Hence, our evaluation outcomes are clinically relevant.
We present both all-cause mortality and non-fatal MI or coronary death as outcomes because these risks differ and the best course of action in terms of risk management and priorities requires evaluation of both mortality and coronary morbidity. Some prognostic factors included in the CALIBER models were much more useful for predicting all-cause mortality than coronary risks and vice versa. Hence, although smoking, diabetes, CVD comorbidities, and biomarkers were good predictors of either outcome, non-CVD comorbidities, depression, and anxiety were mainly useful for predicting non-coronary death. Importantly, the contribution of hypertension to model discrimination was marginal and confined to coronary events. Since blood pressure lowering trials among people with SCAD demonstrate a reduction in both coronary and total mortality rates, the prognostic effect of hypertension may be obscured by treatment.
Widely reported measures of model performance, such as the C-index and the NRI, do not provide clinicians with readily interpretable evidence for making effective, and cost-effective decisions about which patients should be further investigated. Following recommendations, we therefore sought to estimate the potential impact on patient outcomes of using the CALIBER prognostic models for clinical decision making. Using these models to identify patients at high risk (defined by guidelines as 3% annual mortality) and offer a management strategy with a hazard ratio 0.8 (equivalent to ~20% absolute risk reduction) would save an additional 13–16 life years or 15–18 coronary-event-free life years over a 5-year time horizon compared with a model including just age, sex, and social deprivation. Hence, since the models are based on clinically available data (i.e. there is no cost increment of collecting new data), our analysis demonstrates that screening with the models is likely to be cost-effective. However, economic evaluation would be needed to establish their appropriateness in practice.
Stringent external validation is essential to ensure that a prognostic model would be applicable to the rest of the population than the patients in our cohort. However, a model can achieve impressive performance (as high as in the development data) if applied to an external data set that originates from a similar source of data (e.g. validation of QRISK in data from a similar GP database system). To make our external validation rigorous, we therefore used a data set with major differences from the development data set; the data were collected differently (manually abstracted from case records), the patients were at higher risk (received angiography due to chest pain), and the study period was about a decade earlier with markedly different background use of risk lowering medication. Hence, it was reassuring that the models performed well in this external setting. However, as with any prognostic model, there is an ongoing need for further validation in external data sets and recalibration for different populations and time periods.
We propose that the CALIBER prognostic models are implemented in electronic health records in clinical practice. This should be done alongside evaluation of the impact on decisions and, ultimately, patient outcomes. The models may also be used to support adherence to existing therapies, guide frequency of follow-up, plan trials of new therapies, and serve as a reference on which to evaluate the extent to which novel biomarkers might usefully further stratify risk assessment.
Limitations
A limitation in our study is that the electronic health record does not capture all the information available to clinicians prior to imaging. Thus, while clinicians assess symptom severity, this is not recorded in standard fashion. However, we found that a proxy for symptom severity, the use of long-acting nitrates, or repeat prescription for short nitrates did add prognostic value. Although we had information from the resting ECG on heart rate and the presence of atrial fibrillation, resting ST segment, T wave, or other changes were seldom coded. A further limitation was that for external evaluation, we ignored (assumed absent) comorbidities not collected in the external data set. However, the missing predictors had low impact on the performance of the models in the development data; thus, we do not expect this to have affected the evaluation. Further, in order to represent all relevant clinical parameters that are commonly considered in clinical assessment, the models included a large number of variables. However, these variables are routinely recorded in electronic health records, so their collection is not associated with extra costs. Also, the large size of the data set used to develop the models reduces the likelihood of overfitting. Finally, only multiplicative models were considered; models fitted in the additive scale are a plausible alternative that was not explored.
Conclusions
In summary, we present validated prognostic models for estimating risk of all-cause mortality and coronary outcomes based on clinical parameters that are commonly available in all people with stable coronary disease. These models can be implemented alongside further medical investigations to support medical decision making. However, as with any new prognostic model, further independent evaluation is required in different settings including different electronic health record systems, health care organizations, and geographic locations to guide use in clinical practice. A risk calculator is available online (www.caliberresearch.org/model).
Ethics and Registration
We registered the research protocol at clinical trials.gov (NCT01609465) and received approval from the Independent Scientific Advisory Committee (10_160) and the MINAP Academic Group (11-CBR-3).
Source...