Introduction

The CT coronary calcium score (CTCS) is used in both the diagnosis of coronary artery disease (CAD) [1–6] and the prediction of cardiovascular events [7–13]. Although substantial evidence is available on the incremental value of CTCS in predicting future cardiovascular events and mortality in asymptomatic individuals [10–13], the diagnostic value in symptomatic patients is less clear.

A meta-analysis revealed that the presence of coronary calcium had a sensitivity and specificity of 98% and 40%, respectively, in detecting significant stenoses [8]. Furthermore, numerous studies have reported on the value of CTCS in the prediction of the probability of obstructive CAD [1–4]. However, the incremental value (i.e. in addition to all known clinical predictors of CAD) of the CTCS as a continuous predictor of prevalent obstructive CAD is less well studied.

The purpose of this study was to validate previously published clinical prediction models and to determine the incremental value of CTCS for the prediction of prevalent obstructive CAD in patients with new onset stable typical or atypical angina pectoris.

Materials and methods

Study population

The study population was derived from an existing database, which consisted of 402 patients with chest pain suggestive of stable angina pectoris and suspected of having CAD. All patients were prospectively included in a large study evaluating 64-slice CT coronary angiography (CTCA) at our institution. All patients were referred for conventional coronary angiography (CCA) based on their presentation or functional testing that suggested the presence of ischaemia and all patients underwent multidetector CT angiography within a week before CCA. Inclusion criteria for this study were: informed consent, sinus heart rhythm and the ability to hold their breath for 15 s. Patients with a history of percutaneous coronary intervention or coronary artery bypass surgery, impaired renal function (serum creatinine >120 μmol/L) or a known intolerance to iodinated contrast medium were excluded. The Institutional Review Board approved the study and all patients signed informed consent. As this paper focuses on patients with new onset stable chest pain, we also excluded patients with acute coronary syndromes and patients with a previous myocardial infarction (Fig. 1).

Fig. 1
figure 1

Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods

Full size image

CT coronary calcium images

Metoprolol (100 mg, Selokeen, AstraZeneca, London, UK) was administered orally 1 h before CT in patients with heart rates >65 beats per minute. A 64-slice single source CT system (Sensation 64; Siemens, Forchheim, Germany) with a gantry rotation time of 330 ms, acquisition time of 165 ms and voxel size of 0.4 mm3 was used to acquire standard spiral low-dose and ECG-gated coronary calcium CT images. CT parameters were 32 × 2 slices per rotation, individual detector width of 0.6 mm, 3.8-mm/rotation table feed, 120-kV tube voltage, 150-mAs tube current, with activated prospective x-ray tube modulation. Overlapping slices were reconstructed at 65% of the R–R interval using the B35f convolution kernel. Reconstructed slice thickness was 3.0 mm with an increment of 1.5 mm. The radiation exposure, estimated using dedicated software (ImPACT, version 0.99x, St. George’s Hospital, Tooting, London, UK), was 1.4 mSv in men and 1.8 mSv in women. One observer (with more than 3 years’ experience), who was blinded to the CCA and clinical data, measured the coronary calcium.

Conventional coronary angiography

The CTCS and CCA were carried out within 1 week. Coronary segments were assessed on CCA following a 17-segment modified American Heart Association (AHA) classification model [14] by a single observer (with more than 10 years’ experience), who was blinded to the CT and clinical data. A mean luminal narrowing of ≥50% was considered to be a significant stenosis. Validated quantitative coronary angiography software (CAAS II®, Pie Medical, Maastricht, the Netherlands) was used.

Clinical variables and outcome

All patients were interviewed at enrollment in the prospective cohort study. Clinical parameters recorded were: age (years), sex (male/female), type of chest pain (atypical vs. typical), body mass index (BMI) (defined as weight/height2 in kg/m2), smoking status (past or current smoker, yes/no), hypertension (present/absent), dyslipidaemia (serum cholesterol >200 mg/dL or 5.18 mmol/L, present/absent), diabetes (plasma glucose ≥126 mg/dL or 7.0 mmol, present/absent) and family history of CAD (present/absent).

The CTCS was measured by the Agatston method [15] using dedicated software (syngo Calcium Scoring VE31H, Siemens, Germany). The outcome of interest was the presence of obstructive CAD defined as ≥50% stenosis in at least one vessel (present/absent) on CCA.

Sample size

As a general rule, 10 patients with the condition of interest per analysed variable are required for regression analysis. In our dataset (n = 254), 123 patients were identified as having obstructive CAD on CCA. This allowed for the analysis of 12 variables. Our sample meets the required number of cases and non-cases that has been suggested for external validation of prediction models [16].

Systematic literature search

We searched the English-language medical literature in PubMed up to October 14, 2009 for diagnostic prediction models. See the Appendix for a detailed description of the search strategy. From the included articles, clinical variables that were identified as significant predictors of CAD were extracted.

Data analysis

Age was analysed as a continuous variable. To account for the skewed distribution of the coronary calcium scoring, CTCS was transformed by taking the natural logarithm of CTCS + 1. All other variables were dichotomous. Oestrogen status was not available in our dataset. Therefore, we assumed women below the age of 50 to be oestrogen positive, women of 50 years and above to be oestrogen negative and all men to be oestrogen neutral. Obesity was considered in the model by Morise (1997) only. We defined obesity as a BMI >27 kg/m2, corresponding to their definition [17].

The extracted sets of clinical variables were analysed with multivariate logistic regression analysis, fitting new regression coefficients. No attempt was made to validate original regression coefficients, as such coefficients were often not reported. CTCS was subsequently included in each of the models. Models without CTCS were compared with corresponding models including CTCS using the likelihood ratio test. The level of significance was defined at a p value less than 0.05.

Diagnostic performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, the c-index. The c-index is a measure of discrimination and is interpreted as being the probability that a randomly chosen patient with CAD will have a higher predicted probability of disease than a randomly chosen patient without CAD [18]. An area under the ROC curve (AUC) of 0.5 corresponds to a model that provides no diagnostic information, whereas an AUC of 1.0 corresponds to a perfect diagnostic model.

STATA statistical analysis software v10.0 (StataCorp, Texas, USA) was used for logistic regression analysis.

Next, we quantified the effect of adding CTCS to the model on the classification of patients into probability categories of CAD. Four probability categories were defined: <30%, ≥30–50%, ≥50–70% and ≥70%. Reclassification tables were constructed for the Diamond & Forrester model and the Pryor model (see Tables 4 and 5) [19]. We computed the reclassification calibration statistic (RCS) [20] which is equivalent to the Hosmer–Lemeshow statistic, applied to the cross-classified cells of the reclassification table with at least 20 observations. A significant result indicates a lack of fit.

Furthermore, the following reclassification measures were calculated for each model: the overall (correct) percentage of reclassification, the net reclassification improvement (NRI) [21] and the integrated discrimination improvement (IDI) [20]. The NRI is the difference in proportions reclassifying to higher and lower probability categories among cases and non-cases. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. A significant NRI indicates that classification improves when CTCS is included. The IDI compares the difference in the average regression slope of cases and non-cases among the models with and without CTCS. A significant IDI indicates that the new model performs better in discriminating cases and non-cases.

Reclassification computations were executed by using syntax made available by Cook and Ridker [20] in SAS Enterprise Guide v3 (SAS Inc, North Carolina, USA).

Results

Study population

During a 24-month period, 402 patients were enrolled. Patients with acute coronary syndrome or a history of myocardial infarction were excluded and 254 patients were left for data analysis (Table 1). Of these, 123 (48%) patients had obstructive CAD on CCA. CTCS ranged from 0 to 3,839 with a median of 4 for patients without obstructive CAD and a median of 337 for patients with obstructive CAD. Of 131 patients without obstructive CAD, 44 (33.6%) patients had no coronary calcification; whereas of 123 patients diagnosed with obstructive CAD, 3 (2.4%) patients did not have any coronary calcification.

Table 1 Patient characteristics
Full size table

Systematic literature search

We obtained 649 articles in our literature search of which 632 articles were excluded based on title or abstract. After excluding 11 articles based on the full text, 6 studies were left for analysis.

Multivariate logistic regression analysis

Five published prediction rules were validated (Tables 2 and 3): Diamond & Forrester [22], Pryor et al. [23], Morise et al. 1994 [24], Morise et al. 1997 [17] and Shaw et al. [25]. The Diamond & Forrester prediction rule includes age, sex and type of chest pain, all of which were significant predictors of obstructive CAD in our dataset and with an area under the ROC curve (AUC) of 0.798. Including CTCS increased the AUC to 0.890, which was a statistically significant improvement (p < 0.001). In the expanded model age and sex were no longer significant predictors (Table 2).

Table 2 Comparison of multivariate logistic regression models
Full size table
Table 3 Comparison of multivariate logistic regression models
Full size table

Pryor et al. analysed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes and the interaction between age and smoking, age and dyslipidaemia, sex and smoking, and age and sex, of which type of chest pain and the presence of diabetes were significant predictors. This model resulted in an AUC of 0.838. After including CTCS, the AUC increased to 0.901 which was a statistically significant improvement (p < 0.001). In the expanded model diabetes was no longer a significant predictor (Table 2).

Morise et al. (1994) included diabetes and dyslipidaemia in addition to the variables used by Diamond & Forrester and resulted in an AUC of 0.831. All variables were significant predictors of the presence of obstructive CAD. After including CTCS, type of chest pain was the only variable that remained significant. After inclusion of CTCS the AUC increased to 0.899, which was a statistically significant improvement (p < 0.001) (Table 2).

Morise et al. (1997) assessed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes, oestrogen status, hypertension, family history, obesity, BMI and the interaction between dyslipidaemia and family history. This model resulted in an AUC of 0.840. Age, sex, type of chest pain and dyslipidaemia were significant predictors. After including CTCS, the AUC increased to 0.898, which was a significant model improvement (p < 0.001). Age, sex and dyslipidaemia were no longer significant predictors after the addition of CTCS (Table 3).

Shaw et al. considered age, sex, typical chest pain, smoking, dyslipidaemia and diabetes and resulted in an AUC of 0.833. After including CTCS, only type of chest pain remained a significant predictor and the AUC increased to 0.899 which was a statistically significant improvement (p < 0.001) (Table 3).

Reclassification tables for the Diamond & Forrester model and the Pryor model are presented in Tables 4 and 5. The addition of CTCS to Diamond & Forrester resulted in reclassification of 47.2% of patients of whom 73.3% were correctly reclassified. The reclassification calibration statistic (RCS) indicated a strong lack of fit for the Diamond & Forrester model (p < 0.00001) which decreased substantially when CTCS was added to the model (p < 0.01). The NRI (net reclassification improvement) was 33.6% (p < 0.0001) and the IDI (integrated discrimination improvement) was also statistically significant (18.8%, p < 0.001) indicating improvement in the classification of cases and non-cases in probability categories and improvement in discrimination between cases and non-cases.

Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1)
Full size table
Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2)
Full size table

For the model by Pryor et al., 36.2% of the patients were reclassified, of whom 54.3% were correctly classified. The RCS indicated a lack of fit (p = 0.01), which decreased when CTCS was added to the model (p = 0.03). The NRI was 24.0% (p < 0.0001) and the IDI was 14.8% (p < 0.001).

The reclassification measures for all models are presented in Table 6.

Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models
Full size table

Discussion

We analysed the incremental value of CTCS in the prediction of prevalent obstructive CAD. We showed CTCS to be a significant predictor, independent of other variables included in the model. Furthermore, we confirm that the prediction of prevalent obstructive CAD is mainly determined by age, sex and type of chest pain.

In four of the five models age and sex were significant predictors before the addition of CTCS. However, after including CTCS, both age and sex were no longer significant predictors in each of those models. Type of chest pain was a significant predictor of obstructive CAD, independent of other variables included in the model and independent of whether CTCS was included or not. CTCS proved to be an excellent and significant predictor of CAD with adjusted odds ratios close to 2. Apart from age and sex, most risk factors did not result in significant odds ratios which suggests that these risk factors are of minor importance in the prediction of prevalent obstructive CAD. Furthermore, the risk factors that were significant in the models without CTCS lost their significance after including CTCS.

Analysis of reclassification showed that adding CTCS yields reclassification of 34–47% of patients, most of which was correct. The reclassification calibration statistic demonstrated a lack of fit for all models, which decreased when CTCS was added in all models except for Shaw’s.

Numerous studies have previously reported on the incremental value of CTCS in the prediction of cardiovascular events and mortality in asymptomatic individuals [10–13]. Some studies have reported on the incremental value of CTCS in the prediction of prevalent obstructive CAD.

Both Guerci et al. [1] and Kennedy et al. [2] studied the relationship between obstructive CAD and CTCS, adjusting for various risk factors. Both studies found highly significant odds ratios for the CTCS in predicting the presence of obstructive CAD, but the odds ratios were lower compared to what we found. Budoff et al. [4] evaluated the value of CTCS in diagnosing CAD and showed that the addition of CTCS increased the AUC from 0.672 to 0.842. However, the models included only age and sex as clinical variables. Considering the fact that the type of chest pain was not taken into account, an increase in AUC with CTCS can be expected. However, the resulting area under the ROC curve for the model including CTCS was similar to what we found. Schmermund et al. [3] previously studied the value of CTCS in predicting the extent of CAD. They showed an independent and incremental value for CTCS in multiple linear regression in predicting the total number of segments per patient with ≥50% stenosis.

Study limitations

Our study assessed the prediction of ≥50% stenosis in at least one vessel. One could argue that physicians are primarily interested in diagnosing severe CAD, as these patients would be eligible for revascularisation whereas others can be adequately treated medically. Likewise, physicians might be primarily interested in predicting future cardiovascular adverse events. However, we did not consider prognosis in this analysis.

All patients in our study were referred for CCA based on their presentation or functional testing that suggested the presence of cardiac ischaemia. In this way a high-risk population was selected, which could have biased our results. Unfortunately this limitation is inherent to the study design. Further research is necessary to determine the value of CTCS in other (e.g. lower risk) populations.

Also, risk factors such as type of chest pain, smoking status and family history of CAD were obtained by interviewing the patient. Potentially, this method underestimates their predictive effects as compared to the predictive effect of the CTCS, which was directly measured.

Tables 4 and 5 illustrate how CTCS influences the classification of patients in probability categories. However, the limitations of reclassification measures in the context of this research should be taken into account. Our sample size was too small to reliably assess reclassification. For example, the RCS only uses the cross-classified cells containing at least 20 observations. In Table 4 only four cells contain 20 or more observations, implying that a substantial amount of (correctly) reclassified patients are ignored. Thus, in our study the reclassification percentages, NRI and IDI indices are more reliable than the RCS.

Ideally, the probability categories should be based on clinically relevant cutoffs. However, no well-established clinically relevant probability threshold exists. The probability of CAD is commonly defined as low (<30%), intermediate (≥30–70%) and high (≥70%) [26]. In our view the intermediate category is rather wide, which is why we divided this category into low–intermediate (≥30–50%) and intermediate–high (≥50–70%). It should be noted that the overall percentage of reclassification is highly dependent on the choice and number of probability categories.

Clinical implications

Our results demonstrate that the estimation of the probability of obstructive CAD can be improved by including CTCS. This implies that clinicians can make better decisions as to whether a particular patient would benefit from further testing, for example CTCA or CCA. In low-risk patients, a CTCS of 0 could exclude CAD and avoid further testing using CTCA. Hereby, one also avoids the intravenous administration of contrast agent, the extra radiation exposure, and extra scan time and costs associated with CTCA. In patients with a low CTCS, CCA can be avoided and further non-invasive testing would be preferred. In patients with an intermediate CTCS, a CTCA might be the optimal next step. In patients with a high CTCS, direct CCA might be justified because of the high probability of CAD. All in all, CTCS could be useful as a triage test for patients who are suspected of having CAD.

We confirmed that the prediction of significant CAD is primarily driven by the patient’s symptoms. A detailed history of the patient’s symptoms remains most important in the diagnostic work-up of patients with suspected CAD. However, history taking is difficult and subjective, therefore limiting our ability to accurately predict the presence of CAD. Hence, further diagnostic testing will be important, even in patients with a low to intermediate probability of CAD.

On the other hand, the harms and costs of obtaining CTCS should be considered. Kim [27] studied the radiation dose and cancer risk of CTCS screening (every 5 years) in asymptomatic individuals. They concluded that the excess lifetime cancer risk was 42 (62) per 100,000 men (women). It is important to note that our study assessed the value of a single CTCS in symptomatic patients, for whom the excess lifetime cancer risk will be lower and small compared with the risk of missing a CAD diagnosis. Moreover, CTCS could reduce the use of additional testing in patients with a low CTCS and a low probability of CAD, thereby reducing the total radiation exposure.

Although performing a CTCS measurement is a fast, low-dose and relatively inexpensive procedure, the harms and benefits should be considered in a cost-effectiveness analysis.

Conclusion

Our results suggest that CTCS significantly improves the prediction of prevalent obstructive CAD, independent of other clinical variables. Therefore, CTCS should be considered in the diagnostic work-up of CAD.