We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Net Risk Reclassification P Values: Valid or Misleading?
- Authors
Pepe, Margaret S.; Janes, Holly; Li, Christopher I.
- Abstract
Background The Net Reclassification Index (NRI) and its P value are used to make conclusions about improvements in prediction performance gained by adding a set of biomarkers to an existing risk prediction model. Although proposed only 5 years ago, the NRI has gained enormous traction in the risk prediction literature. Concerns have recently been raised about the statistical validity of the NRI. Methods Using a population dataset of 10000 individuals with an event rate of 10.2%, in which four biomarkers have no predictive ability, we repeatedly simulated studies and calculated the chance that the NRI statistic provides a positive statistically significant result. Subjects for training data (n = 420) and test data (n = 420 or 840) were randomly selected from the population, and corresponding NRI statistics and P values were calculated. For comparison, the change in the area under the receiver operating characteristic curve and likelihood ratio statistics were calculated. Results We found that rates of false-positive conclusions based on the NRI statistic were unacceptably high, being 63.0% in the training datasets and 18.8% to 34.4% in the test datasets. False-positive conclusions were rare when using the change in the area under the curve and occurred at the expected rate of approximately 5.0% with the likelihood ratio statistic. Conclusions Conclusions about biomarker performance that are based primarily on a statistically significant NRI statistic should be treated with skepticism. Use of NRI P values in scientific reporting should be halted.
- Subjects
BIOCHEMISTRY experiments; BIOMARKERS; STATISTICAL hypothesis testing; PROBABILITY theory; QUANTITATIVE research; NULL hypothesis
- Publication
JNCI: Journal of the National Cancer Institute, 2014, Vol 106, Issue 4, p1
- ISSN
0027-8874
- Publication type
Article
- DOI
10.1093/jnci/dju041