Positive Behavior Interventions and Supports (PBIS) in Challenging Contexts: Evaluating a Replicable Implementation Approach in Philadelphia, Pennsylvania, 2018-2021 (ICPSR 38572)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Data Collection Notes View help for Data Collection Notes

Study Purpose View help for Study Purpose

The goals of the study were to assess the effectiveness of two different interventions with different intensities and resource requirements:

1. Brief trauma trainings for school safety officers. This series of trainings was delivered to school officers in the treatment group during the 2018-2019 and 2019-2020 school years. The impacts of this intervention were assessed via an RCT, where officers from eligible schools were randomly assigned to either receive the trauma trainings or to continue with business-as-usual. Approximately half of the schools in each condition were participants in the partnering school district's PBIS initiative.
2. Expanded climate supports, including intensive whole-staff training and ongoing coaching in trauma and schoolwide PBIS; universal screening for students; and expanded support for Tier 2 implementation. The impacts of this implementation model were assessed via a cluster randomized experiment, with the intervention delivered to classrooms nested in 4 PBIS schools randomly selected from a list of 9 eligible schools. The remaining 5 PBIS schools served as a comparison group.

The study addressed the following objectives:

1. Assess the impacts of trauma-awareness trainings for school safety officers in K-8 schools in the partnering school district on school officers': 1) knowledge and attitudes regarding trauma and trauma-informed practices, and 2) reported behaviors (i.e., use of trauma-informed practices) in their interactions with students.
2. Assess the implementation (including acceptability, appropriateness, and feasibility) and impacts of an expanded, trauma-informed PBIS implementation model in 4 K-5 or K-8 schools on teacher outcomes and student outcomes in the domains of attendance, discipline, and academics, relative to a business-as-usual comparison group of schools.
3. Assess the costs of the expanded, trauma-informed PBIS implementation model components.

Study Design View help for Study Design

The impacts of trauma training for school officers were assessed via an random controlled trial (RCT) of all eligible district K-8 schools, including schools that did and did not use school-wide Positive Behavior Interventions and Supports (PBIS). The RCT used a blocked design to assign schools to treatment and control conditions within PBIS status (i.e., PBIS school, non-PBIS school). Impacts of the implementation model were assessed via a cluster randomized experiment. To select schools to receive the expanded supports being provided through this project, the research team collaborated with the partnering school district's school climate administrators and project partners who had been supporting PBIS implementation in the district for several years prior to the start of the project.

Criteria were established to determine eligibility for the implementation model, including that eligible schools must have: 1) strong PBIS implementation at Tier 1, 2) attempted implementing some form of Tier 2 supports, and 3) leadership buy-in. Applying these criteria, nine potential schools were identified as eligible. From this list, four schools were randomly selected to receive the implementation model and the remaining five schools served as Comparison schools and conducted business-as-usual PBIS.

The research team collected survey data from K-5 teachers at Demonstration and Comparison schools in the Fall and Spring of the 2019-20 and 2020-21 school years, as well as conducted semi-structured interviews with Demonstration school stakeholders in Spring 2020 and Spring 2021. Survey and interview data from Demonstration school teachers collected each Spring were used to assess the implementation of the expanded, trauma-informed PBIS implementation model. Additionally, survey data from Demonstration and Comparison school teachers across time-points were used to examine changes in teachers' beliefs, self-efficacy, intentions, and self-reported behaviors

Administrative data was also requested and received from the partnering school district at the classroom level for the four Demonstration and five Comparison schools. These data were used to make comparisons between the two sets of schools on a range of student demographic, attendance, climate, disciplinary, and academic outcomes

Sample View help for Sample

School Safety Officer Survey Dataset: Participants (N = 97) most often identified as male (54%), Black or African American (65%), and between the ages of 30 and 60 years (82%).

Teacher Survey Dataset: The teacher survey was administered to teachers in both Demonstration and Comparison schools in Fall and Spring of the 2019-20 and 2020-21 school years. At each time point, all K-5 teachers in the Demonstration and Comparison schools were invited by email to participate in the voluntary, online survey. Across both timepoints, 301 surveys were sent, and we received a total of 180 responses, a rate of 59.8%.

Administration Dataset: Administrative data was provided by the school district for all K-5 students, aggregated at the classroom level, within the nine participating schools (4 Demonstration, 5 Comparison) for the academic years 2016-17, 2017-18, 2018-19, 2019-20, and 2020-21.

Time Method View help for Time Method

Universe View help for Universe

School safety officer random controlled trial (RCT) surveys: Any officer assigned to a study school (treatment or control) between 2018 and 2020. Implementation model RCT surveys: All K-5 teachers in treatment and comparison schools.

Implementation model RCT interviews: All principals and coaches in treatment schools; teachers from treatment schools representing different grade levels.

Unit(s) of Observation View help for Unit(s) of Observation

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Response Rates View help for Response Rates

School Safety Officer Survey Dataset: N/A

Administration Dataset: Spring 2020 - 55.4%; Spring 2021 - 70.5%

Teacher Survey Dataset: Fall 2019 - 62.3%; Spring 2020 - 70.8%; Fall 2020 - 70.8%; Spring 2021 - 68.2%

Original Release Date View help for Original Release Date

2024-05-15

Version History View help for Version History