A Roadmap to Evidence-Based School Safety: Safe Communities Safe Schools, Colorado, 2016-2020 (ICPSR 37913)

Version Date: Mar 29, 2023

Cite this study |

Principal Investigator(s):
Beverly E. Kingston, University of Colorado, Boulder; Sabrina Arredondo Mattson, University of Colorado, Boulder; Allison Dymnicki, Abt Associates; Elizabeth Spier, American Institutes for Research

https://doi.org/10.3886/ICPSR37913.v1

Version V1

Slide tabs to view more

Summary

Researchers from the University of Colorado Boulder's Center for the Study and Prevention of Violence (CSPV) partnered with educators in 46 middle schools to implement Safe Communities Safe Schools (SCSS). SCSS seeks to prevent and reduce behavioral incidents, address mental and behavioral health concerns, and increase prosocial behavior in the school setting through three core program components: developing a functioning multidisciplinary school team, building capacity around data use, and selecting and implementing evidence-based programs. The study explored research questions in three areas: readiness (whether schools met baseline criteria and experienced changes in readiness over time), implementation (whether the SCSS model was implemented as intended; whether it is feasible, acceptable, and effective when implemented schoolwide), and associated outcomes (effects on school climate, safety, related behavioral and mental health indicators, and academic outcomes). To explore questions in these three areas, CSPV and external evaluators from American Institutes for Research conducted a mixed-methods randomized control trial with a staggered implementation design using qualitative data (open-ended questions on implementation surveys, focus groups) and quantitative data (staff and student school climate data, attendance/truancy rates, and suspension rates, and academic achievement data).

This collection is organized into 12 parts and includes administrative school record data, student and staff climate surveys, and fidelity data. School record data from years 1 and 2 of the study include school-level attendance, truancy, and suspension rates, as well as student-level assessment data. Qualitative focus group data is not currently included in the collection.

Citation

Kingston, Beverly E., Arredondo Mattson, Sabrina, Dymnicki, Allison, and Spier, Elizabeth. A Roadmap to Evidence-Based School Safety: Safe Communities Safe Schools, Colorado, 2016-2020. Inter-university Consortium for Political and Social Research [distributor], 2023-03-29. https://doi.org/10.3886/ICPSR37913.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

United States Department of Justice. Office of Justice Programs. National Institute of Justice (2105-CK-BX-K002)

Subject Terms

Geographic Coverage

Smallest Geographic Unit

None.

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2016 -- 2020

Date of Collection

2016 -- 2019 (School Records and Climate Survey Data), 2017 -- 2020 (Mid-Implementation Survey)

Data Collection Notes

  1. The qualitative focus group data collected for this study are not available as part of the collection at this time.

Hide

Study Purpose

Researchers from the University of Colorado Boulder's Center for the Study and Prevention of Violence (CSPV) partnered with educators in 46 middle schools to implement Safe Communities Safe Schools (SCSS), which seeks to prevent and reduce behavioral incidents, address mental and behavioral health concerns, and increase prosocial behavior in the school setting. The study addressed four broad research questions:

  1. Do schools have initial readiness and increase their readiness to implement the SCSS model over time?
  2. Is the SCSS model implemented as intended in this set of schools?
  3. Is the SCSS model feasible, acceptable, and effective when implemented schoolwide?
  4. What is the effect on school climate, safety, related behavioral and mental health indicators, and academic outcomes for youth and staff in schools randomized to receive the SCSS model, compared to youth and staff in schools randomized to the waitlist control group; and how do these impacts vary by variation in implementation?

Study Design

The research team conducted a staggered-implementation randomized control trial. Project subjects included educators at 46 middle schools, the students in these schools, and (indirectly) their families and broader community. Cohort 1 included 10 (5 treatment schools and 5 control schools) schools and Cohort 2 included 36 schools (18 treatment schools and 18 control schools). Schools were assessed for initial readiness and were supported to increase capacity to implement Safe Communities Safe Schools (SCSS) throughout the project period. Groups of schools within each cohort began receiving SCSS in different years so that all schools received at least 1 year of SCSS treatment by the project's end. The study examined outcomes only at years 1 and 2.

To address issues related to readiness, University of Colorado Boulder's Center for the Study and Prevention of Violence (CSPV) team members conducted readiness feasibility visits for recruitment, and school-based teams completed a 90-item readiness assessment annually. Six types of implementation data were collected and analyzed to address implementation fidelity: (1) SCSS training documents, (2) mid-year and (3) end-of-year surveys completed by program facilitators of the school's selected evidence-based program, (4) observations of classroom-based evidence-based programs, (5) implementation ratings completed by CSPV implementation managers, and (6) structured focus groups with school-based teams. Several types of outcome data were collected and analyzed to address SCSS impacts on student and school outcomes, including student and staff climate survey data assessing risk and protective factors and violence indicators, and school record data assessing daily attendance and truancy rates, suspension rates, and academic achievement test scores (reading and math).

Sample

Schools were recruited from the Colorado Front Range. To establish baseline equivalence, schools were matched using optimal paired matching and then randomized into treatment or control conditions. Five variables were used to find the best matches for each school: (1) site selection ratings, (2) enrollment numbers, (3) percentage non-white students, (4) percentage of students receiving free or reduced price lunch, and (5) average standardized proficiency rate across subjects (math, reading, and writing) for grades 6-8. Subsequent analyses done on the treatment and comparison groups further indicated that these were valid matches, based on similar group means on the five variables included in the matching process plus additional school characteristics (e.g., percentage of students Hispanic, percent of students African American, mobility rates, and attendance rates). All cases were used in each school in subsequent analyses.

Time Method

Longitudinal: Cohort / Event-based

Universe

Middle school students and staff from participating schools in the Colorado Front Range.

Unit(s) of Observation

School, Individual

Data Source

School record data was obtained from the Colorado Department of Education.

Data Type(s)

Mode of Data Collection

Response Rates

Please see Appendix A of the School Climate Survey Protocol documentation for student and staff climate survey response rates, and Attachment A of the Mid-Implementation Survey Protocol documentation for mid-implementation survey response rates.

Presence of Common Scales

The student and staff climate were modified version of the Youth Risk Behavior Survey. Several Likert-type scales were used in these.

The mid-implementation survey was modified from existing implementation and fidelity surveys used for several programs, including Positive Action, Olweus Bullying Prevention Program, and Achievement Mentoring. Several Likert-type scales were used in these, as well as some open-ended questions.

Hide

Original Release Date

2023-03-29

Version History

2023-03-29 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Checked for undocumented or out-of-range codes.

Hide

Weight

Not applicable.

Hide

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.