We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Assessing discriminative ability of risk models in clustered data.
- Authors
van Klaveren, David; Steyerberg, Ewout W.; Perel, Pablo; Vergouwe, Yvonne
- Abstract
Background The discriminative ability of a risk model is often measured by Harrell's concordance-index (c-index). The c-index estimates for two randomly chosen subjects the probability that the model predicts a higher risk for the subject with poorer outcome (concordance probability). When data are clustered, as in multicenter data, two types of concordance are distinguished: concordance in subjects from the same cluster (within-cluster concordance probability) and concordance in subjects from different clusters (between-cluster concordance probability). We argue that the within-cluster concordance probability is most relevant when a risk model supports decisions within clusters (e.g. who should be treated in a particular center). We aimed to explore different approaches to estimate the within-cluster concordance probability in clustered data. Methods We used data of the CRASH trial (2,081 patients clustered in 35 centers) to develop a risk model for mortality after traumatic brain injury. To assess the discriminative ability of the risk model within centers we first calculated cluster-specific c-indexes. We then pooled the cluster-specific c-indexes into a summary estimate with different meta-analytical techniques. We considered fixed effect meta-analysis with different weights (equal; inverse variance; number of subjects, events or pairs) and random effects meta-analysis. We reflected on pooling the estimates on the log-odds scale rather than the probability scale. Results The cluster-specific c-index varied substantially across centers (IQR = 0.70-0.81; I2 = 0.76 with 95 % confidence interval 0.66 to 0.82). Summary estimates resulting from fixed effect meta-analysis ranged from 0.75 (equal weights) to 0.84 (inverse variance weights). With random effects meta-analysis - accounting for the observed heterogeneity in c-indexes across clusters - we estimated a mean of 0.77, a between-cluster variance of 0.0072 and a 95 % prediction interval of 0.60 to 0.95. The normality assumptions for derivation of a prediction interval were better met on the probability than on the log-odds scale. Conclusion When assessing the discriminative ability of risk models used to support decisions at cluster level we recommend meta-analysis of cluster-specific c-indexes. Particularly, random effects meta-analysis should be considered.
- Subjects
BRAIN injuries; COMPLICATIONS of brain injuries; PATIENTS; META-analysis; CONCORDANCES; QUANTITATIVE research
- Publication
BMC Medical Research Methodology, 2014, Vol 14, Issue 1, p1
- ISSN
1471-2288
- Publication type
Article
- DOI
10.1186/1471-2288-14-5