How Well Do Clinical Prediction Models (CPMs) Validate? A Large-Scale Evaluation of Cardiovascular Clinical Prediction Models [Methods Study], United States, 2016-2021 (ICPSR 39624)

Version Date: Dec 15, 2025

Cite this study |

Principal Investigator(s):
David M. Kent, Tufts Medical Center

https://doi.org/10.3886/ICPSR39624.v1

Version V1

Slide tabs to view more

Summary

Clinical prediction models, or CPMs, are statistical models that can predict a patient's risk for a specific event, such as a health problem, adverse effect, or even death. To create a CPM, researchers use a single data set, such as data from a clinical trial. To find out whether the CPM accurately predicts risks for patients who weren't part of the original data, researchers can test the CPM with other data sets. This testing can help researchers know if the CPM is accurate for patients from different backgrounds and whether it can be used to make health decisions. But few CPMs have been tested with other data sets.

In this study, the research team used other data sets to look at how well CPMs for heart disease predict patients' risks. They also looked at how to improve CPMs.

To access the software and methods, please visit the Tufts Race CPM Registry.

Citation

Kent, David M. How Well Do Clinical Prediction Models (CPMs) Validate? A Large-Scale Evaluation of Cardiovascular Clinical Prediction Models [Methods Study], United States, 2016-2021. Inter-university Consortium for Political and Social Research [distributor], 2025-12-15. https://doi.org/10.3886/ICPSR39624.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

Patient-Centered Outcomes Research Institute (PCORI) (ME-1606-35555)

Subject Terms

Geographic Coverage

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2016 -- 2021
Hide

Study Purpose

(1) To understand how well cardiovascular Clinical prediction models (CPMs) perform using data from external cohorts; (2) To test the effectiveness of updating procedures to improve risk predictions from CPMs

Study Design

The research team first conducted a systematic review of 1,382 Clinical prediction models (CPMs) to assess how frequently CPMs are validated, and performance variation across external cohorts. Then the team used 36 clinical data sets to validate 108 CPMs for three health conditions: acute coronary syndrome, heart failure, and incident cardiovascular disease. To test CPM performance, the team used three metrics:

  • Discrimination, or how well a CPM separates patients with the outcome from those without the outcome.
  • Calibration error, or the difference between the observed outcome rate and the estimated probabilities.
  • Net benefit, or a measure of whether basing clinical decisions on the CPM would do more good than harm at different risk thresholds. The risk thresholds were the observed rate of an outcome, twice the observed rate, and half the observed rate. The research team tested three statistical updating procedures to improve CPM predictions.

With input from a stakeholder panel that included researchers, clinicians, industry experts, and patient advocates, the research team created a website with a comprehensive database of each CPM's performance.

Data Source

Tufts Predictive Analytics and Comparative Effectiveness Center (PACE) CPM Registry Publicly available clinical trial data from the National Heart, Lung, and Blood Institute

Hide

Original Release Date

2025-12-15

Hide

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • ICPSR usually offers files in multiple formats for researchers to be able to access data and documentation in formats that work well within their needs. If you have questions about the accessibility of materials distributed by ICPSR or require further assistance, please visit ICPSR’s Accessibility Center.