Develop, Test, and Disseminate a New Technology to Modernize Data Abstraction in Systematic Reviews [Methods Study], United States, 2013-2019 (ICPSR 39615)

Version Date: Dec 15, 2025

Cite this study |

Principal Investigator(s):
Tianjing Li, Johns Hopkins University

https://doi.org/10.3886/ICPSR39615.v1

Version V1

Slide tabs to view more

Summary

Systematic reviews combine the results of many studies. In health research, these reviews can help determine which treatments or types of care work best. As part of a systematic review, researchers find and record important study information, such as design and results, from published journal articles. This process, called abstraction, takes time. If researchers make errors during this process, the systematic review may come to incorrect conclusions, which can affect healthcare decisions. Researchers abstract information in different ways. In single abstraction and verification, one person abstracts information and a second person reviews it for accuracy. In dual abstraction, two people abstract information on their own and compare the results.

In this study, the research team created and tested a new software program to help with abstraction. In the new software, researchers place flags within a journal article, displayed next to a data collection form on a computer screen, to easily find abstracted information. The team compared three approaches for abstracting information:

  • Single abstraction and verification with the new software
  • Single abstraction and verification without the new software
  • Dual abstraction without the new software

The research team looked at how accurate the abstractions were and how much time it took to do them.

To access the software and methods, please visit the DAA Bitbucket.

Citation

Li, Tianjing. Develop, Test, and Disseminate a New Technology to Modernize Data Abstraction in Systematic Reviews [Methods Study], United States, 2013-2019. Inter-university Consortium for Political and Social Research [distributor], 2025-12-15. https://doi.org/10.3886/ICPSR39615.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

Patient-Centered Outcomes Research Institute (PCORI) (ME-1310-07009)

Subject Terms

Geographic Coverage

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2013 -- 2019
Hide

Study Purpose

To compare the accuracy and efficiency of data abstraction supported by new Data Abstraction Assistant (DAA) software with two traditional methods for data abstraction in systematic reviews.

Study Design

This randomized crossover trial compared the accuracy and efficiency of single abstraction plus verification using DAA (DAA verification), single abstraction plus verification without DAA (regular verification), and independent dual abstraction without DAA (independent abstraction). The research team developed DAA for this study; DAA creates a permanent link between relevant information in the journal article and the electronic data collection form displayed side-by-side on a computer screen used for abstraction.

The study included 52 participants with prior data abstraction experience. Participants worked in 26 pairs; each pair included a more experienced abstractor who had authored three or more published systematic reviews and a less experienced abstractor. Each pair abstracted six articles on different topics, such as preventing falls or depression treatment. Pairs used each of the three approaches, in a random order, on two articles:

  • DAA verification. The less experienced abstractor used DAA to complete abstractions. The more experienced abstractor reviewed for accuracy and, if needed, changed the abstracted content or discussed changes with the original abstractor to reach consensus.
  • Regular verification. The less experienced abstractor completed abstractions without using DAA. As with DAA verification, the more experienced abstractor reviewed for accuracy and worked with the original abstractor to reach consensus.
  • Independent abstraction. Each of the two abstractors worked independently without using DAA. The pair then compared responses and discussed any differences to reach consensus.
  • To determine error proportions, the research team compared abstracted information with an answer key developed by two experienced systematic reviewers. The abstractors also recorded the time spent on data abstraction.

Patients, policy makers, health industry representatives, and researchers with experience in systematic reviews, clinical trials, and clinical practice guidelines contributed to all aspects of the study.

Universe

Researchers from Johns Hopkins University and Brown University

Data Source

Data items abstracted from journal articles by 52 participants ages 20 and older with data abstraction experience

Hide

Original Release Date

2025-12-15

Hide

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • ICPSR usually offers files in multiple formats for researchers to be able to access data and documentation in formats that work well within their needs. If you have questions about the accessibility of materials distributed by ICPSR or require further assistance, please visit ICPSR’s Accessibility Center.