Evaluation of the P2P Challenging Extremism Initiative, Massachusetts and Utah, 2016-2019 (ICPSR 37338)

Version Date: Apr 27, 2020 View help for published

Principal Investigator(s): View help for Principal Investigator(s)
Elena Savoia, Harvard School of Public Health

https://doi.org/10.3886/ICPSR37338.v1

Version V1

This project convened experts and practitioners in the areas of program evaluation, radicalization to violent extremism, and social media analytics in order to generate and integrate scientifically derived knowledge into strategies for effective prevention and intervention against domestic radicalization and violent extremism in the United States. More specifically, we generated substantive evaluation data, which can be used by practitioners and policy makers to enhance the creation and dissemination of effective counter-narratives for reducing the threat of ideologically-motivated violence in the US. We used a mixed-methods approach to evaluate an existing nationwide initiative, Peer-to-Peer (P2P): Challenging Extremism, which aims at engaging youth in countering violent extremism in schools and online arenas.

The project had four specific objectives: 1) Evaluate the content and dissemination of the P2P Initiative social media products, 2) evaluate the impact of the P2P Initiative on youth engaged in its development, 3) evaluate the impact of youth exposure to the P2P educational activities, and 4) assess the drivers of success and barriers in the implementation of the initiative.

To complete these objectives, the following research phases were conducted:

A secondary review of 150 P2P social media products created between fall 2015 and spring 2017, including data on end-users interactivity.

Phone and in-person group interviews with faculty and students engaged in the P2P Initiative.

A prospective cohort study evaluating the impact of the Kombat with Kindness (KWK) campaign on Utah secondary school students, using a pre-post intervention design.

A randomized control study evaluating the impact of the Operation 250 (OP250) on Massachusetts secondary school students, using a pre-post intervention design.

Phone interviews with faculty who implemented the P2P Initiative.

Savoia, Elena. Evaluation of the P2P Challenging Extremism Initiative, Massachusetts and Utah, 2016-2019. Inter-university Consortium for Political and Social Research [distributor], 2020-04-27. https://doi.org/10.3886/ICPSR37338.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote
United States Department of Justice. Office of Justice Programs. National Institute of Justice (2016-ZA-BX-K001)

None

Access to these data is restricted. Users interested in obtaining these data must complete a Restricted Data Use Agreement, specify the reason for the request, and obtain IRB approval or notice of exemption for their research.

Inter-university Consortium for Political and Social Research
2016-01-01 -- 2019-05-31
2016-06-01 -- 2019-05-09
  1. Only the data from the cohort study and randomized control study are available at this time.

The purpose of the project was to independently evaluate an existing nationwide initiative, Peer-to-Peer (P2P): Challenging Extremism, which aims at engaging youth in countering violent extremism in schools and online arenas. By determining the reach, effectiveness, successes, and barriers of the program's implementation, practitioners and policy makers can enhance the creation and dissemination of effective counter-narratives for reducing the threat of ideologically-motivated violence in the US.

The project had four specific objectives: 1) Evaluate the content and dissemination of the P2P Initiative social media products, 2) evaluate the impact of the P2P Initiative on youth engaged in its development, 3) evaluate the impact of youth exposure to the P2P educational activities, and 4) assess the drivers of success and barriers in the implementation of the initiative.

Web content review: 150 P2P products created 2015-2017 were categorized by campaign goals, target audience, social media metrics, and number of in-person events held. This was used to determine which P2P-created campaigns to evaluate, based on impact on target audience.

Interviews with youth in the Initiative: 24 phone interviews and 4 in-person group interviews were conducted with faculty and students who were former participants in the P2P Initiative, specifically regarding their ability to develop counter-narratives or discredit extremist propaganda. Examples of propaganda narratives were obtained through web scraping of a white nationalist forum.

Kombat with Kindness intervention: Researchers worked with the KWK initiative to identify schools that had implemented its materials. Two Utah middle schools were selected as the intervention condition (participated in KWK), while a third non-implementing school that was demographically similar was selected as the control condition (received materials on preparing for weather-related emergencies). 8th and 9th grade students who participated were given pretest and posttest surveys.

Operation 250 intervention: Students at two Massachusetts high schools were selected for participation in the study. Participants were randomly assigned to receive educational materials related to Operation 250 (intervention condition) or to preparing for weather-related emergencies (control condition). Pretest and posttest surveys at 1 month and 2 months were distributed.

Interviews with faculty implementers: Faculty and staff engaged in the P2P Initiative across the United States were recruited for phone interviews. Questions focused on reasons for joining the initiative, organizational and logistical issues, and recommendations for future initiatives.

Intervention study participants were recruited by schools and community-based organizations via convenience methods. OP250 study participants were randomly assigned to a condition, while KWK participants exposed to educational materials were matched with students in the control condition.

Former youth participants were purposively sampled for interviews, while faculty participants who conducted phone interviews were recruited via snowball and convenience sampling.

Longitudinal: Cohort / Event-based, Cross-sectional, Longitudinal: Trend / Repeated Cross-section

For interviews: Youth between 14-25 years old and university faculty across the United States who participated in the P2P Initiative.

For intervention studies: Youth between 14-21 years old living in Massachusetts or Utah.

For web content review: Social media products created for the P2P Initiative between fall 2015 and spring 2017.

Individual

In both intervention studies, items included participant demographics (age, grade level, gender, race), experiences of being treated unfairly, exposure to insulting and accepting messages, experiences of unsafe situations online, and social media use (types, frequency of use).

Items unique to the KWK study were related to cultural sensitivity and empathy (e.g. I enjoy interacting with kids from different cultures), and awareness of racial/ethnic oppression. Intervention participants evaluated KWK's learning objectives; control participants were asked to list emergency supplies needed for a winter storm.

Items unique to the OP250 study were related to comfort with Internet/technology use, risk assessment of online behaviors, knowledge about bias and hate, and online safety.

Response rates ranged from 85% to 95%.

Cultural Intelligence Scale (Ang et al. 2007)

Scale of Ethno-Cultural Empathy (Wang et al. 2003)

Perception of racism in children and youth scale (Pachter et al. 2010)

2020-04-27

2020-04-27 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Checked for undocumented or out-of-range codes.

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.

  • The citation of this study may have changed due to the new version control system that has been implemented.