Net reclassification improvement and integrated discrimination improvement require calibrated models: Relevance from a marker and model perspective

Introduction
For the last three decades, clinical prediction models have mainly been evaluated on the basis of their ability to discriminate between persons who develop the event of interest and persons who do not, as quantified by the c-statistic or area under the receiver operator characteristic curve (AUC). The AUC considers sensitivity and specificity of the model over all possible cut-points of predicted risk. However, prediction models are often used to classify patients into risk categories that correspond to diagnostic or therapeutic decisions. This provoked the idea of comparing models according to their ability to adequately assign clinical risk categories based on absolute risk estimates. Analyses of risk reclassification have hit the ground running: uptake of measures such as net reclassification improvement (NRI) has been enormous, and guidance documents on evaluations of markers and prediction models embraced it as a step prior to full-blown cost-effectiveness analysis.More recently, several researchers reviewed the current applications of reclassification analysis and expressed concerns about inappropriate use.