Dear Editor,
With interest we read the article by Raney et al. [1] in which they developed a prediction rule to identify patients with neck pain who are likely to improve with cervical traction. We agree with the authors that external validation is needed before the rule can be implemented in routine clinical practice. However, we doubt that this rule will be valid in an external population, because some analytical choices made by the authors probably hamper external validity.
First, they did not apply the rule of thumb of at least ten events per variable [2], which introduces bias into the tests of significance. The exact number of variables is not mentioned, but exceeds the advised amount of three variables (30 events) by far. Second, continuous variables were dichotomized based on their optimal cutoff point. This inflates the type I error rate and may result in wrongful identification of a variable as prognostically relevant [3]. Furthermore, categorisation of continuous variables before entering them into a stepwise logistic regression selection procedure may also result in the wrong predictors in the final model [4], and a poorer performance of the model [4, 5].
These analytical choices finally lead to the prediction rule through a stepwise logistic regression procedure, of which the selection criteria are not mentioned. The characteristics of the prediction rule are presented in Table 3 and 4 [1]. However, we would like to point out that Table 4 contains several incorrect values (e.g. sensitivity = 0.83, instead of 0.30, and specificity = 0.42, instead of 0.97, if ≥2 predictors are present) [1].
In conclusion, we think that this prediction rule most likely only applies to the development population and will lack external validity.
References
Raney NH, Petersen EJ, Smith TA, Cowan JE, Rendeiro DG, Deyle GD, Childs JD (2009) Development of a clinical prediction rule to identify patients with neck pain likely to benefit from cervical traction and exercise. Eur Spine J 18:382–391
Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR (1996) A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 49:1373–1379
Altman DG, Lausen B, Sauerbrei W, Schumacher M (1994) Dangers of using “optimal” cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst 86:829–835
Schellingerhout JM, Heymans MW, de Vet HC, Koes BW, Verhagen AP (2009) Categorizing continuous variables resulted in different predictors in a prognostic model for nonspecific neck pain. J Clin Epidemiol 62:868–874
Royston P, Altman DG, Sauerbrei W (2006) Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 25:127–141
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Schellingerhout, J.M., Verhagen, A.P. Letter to the Editor concerning “Development of a clinical prediction rule to identify patients with neck pain likely to benefit from cervical traction and exercise” by Raney N et al. (2009) Eur Spine J 18:382–391. Eur Spine J 19, 833 (2010). https://doi.org/10.1007/s00586-010-1322-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00586-010-1322-0