Developing and Validating Quantitative Measures to Assess Community Engagement in Research: Addressing the Measurement Challenge, United States, 2017-2020 (ICPSR 38493)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Study Purpose View help for Study Purpose

To develop and validate comprehensive and condensed versions of a survey instrument that will be completed by stakeholders and used to assess engagement in research studies from a stakeholder perspective.

Study Design View help for Study Design

A literature review on community engagement in research was conducted to identify existing measures of engagement. Using the engagement principles (EPs), a comprehensive community engagement measure was identified and developed, and its construct validity was then evaluated. The Disparities Elimination Advisory Committee (DEAC), a community-academic partnership at the Siteman Cancer Center and the Washington University in St. Louis School of Medicine, served as the community advisory board guiding the work. The DEAC and the Patient Research Advisory Board (PRAB), a community group trained in research methods, served as community partners and assisted in determining the components to include in the measurement tool: engagement categories and EPs. These partners also advised on the development of survey items that would demonstrate the presence of EPs and on the recruitment and retention of participants to use the measurement tool.

To assess content validity, a panel of national experts in stakeholder-engaged research (SER) were convened, and used a 5-round Delphi process to reach agreement on engagement categories and their definitions, EPs and their definitions, and items associated with each EP. The DEAC and the PRAB provided feedback on the work of the Delphi panel. The expert review panel comprised a range of stakeholders (e.g., patients, caregivers, advocacy groups, clinicians, researchers).

After initiation of the Delphi process, community members who had participated in patient- or community-engaged research across the community engagement spectrum were recruited to participate in a longitudinal, web-based study that consisted of 4 surveys over an 18-month period. As the assessment tool was developed to measure categories of community engagement, the surveys permitted the investigators to examine the reliability and validity of the tool, which was named the Research Engagement Survey Tool (REST). The performance of each item on the tool was also evaluated.

Two types of psychometric properties of the survey items were examined: internal consistency and convergent validity with other scales (e.g., Trust in Medical Researchers Scale). After the Delphi process and between Surveys 3 and 4, cognitive interviews about the survey items to improve the reliability and validity of REST were completed.

REST has two scoring approaches. The first approach, aligned with EPs, is scored by taking the mean of the 3 to 5 items for each EP to obtain EP-specific scores and then taking the overall mean of the EP means to obtain an overall REST score. The second scoring approach aligns REST with the categories of engagement, developed during the Delphi process, based on the percentage of item responses that are aligned with each of 5 categories: (1) outreach and education, (2) consultation, (3) cooperation, (4) collaboration, and (5) partnership. The scoring approaches were applied to the 32-item and 9-item versions of the REST.

Finally, the implementation of the final version of REST (long form) in 20 project teams were studied. These 20 project teams worked alongside the study to administer REST to their stakeholders in 26 separate PCOR and SER projects; the project teams completed surveys and interviews that examined barriers to and facilitating factors for implementing REST.

Sample View help for Sample

Convenience, snowball sampling methods.

Time Method View help for Time Method

Universe View help for Universe

Adults (age 18 or older) involved in community-engaged research.

Unit(s) of Observation View help for Unit(s) of Observation

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Description of Variables View help for Description of Variables

A panel of national experts in SER were convened, and a 5-round Delphi process was applied to reach agreement on engagement categories and their definitions, EPs and their definitions, and items associated with each EP. After initiation of the Delphi process, community members who had participated in patient- or community-engaged research across the community engagement spectrum were recruited to participate in a longitudinal, web-based study that consisted of 4 surveys over an 18-month period.

The variables present in these data represent results from these efforts.

Original Release Date View help for Original Release Date

2022-09-19