A team of researchers from SRI International and Copia Consulting (hereafter referred to as the SRI team) partnered with the Pharr-San Juan-Alamo Independent School District (PSJA ISD) as an external evaluator, per grant requirements, to answer the following research questions:
- To what extent was the Foundations program implemented with fidelity in the participating schools? What factors facilitated or constrained implementation and why?
- Was implementation of the Foundations program effective at improving students' perceptions of school climate, safety, and school engagement; increasing student attendance; increasing students' academic achievement; and reducing students' problem behaviors and discipline referrals?
- Was implementation of the Foundations program effective at improving staff perceptions of school climate and safety?
The SRI team assessed the district's degree of implementation of Foundations and its associated outcomes at the school, student, and staff levels in participating schools. This randomized controlled trial (RCT) had a mixed-methods design in which the team collected and analyzed both quantitative data (e.g., administrative data and surveys) and qualitative data (i.e., interviews, focus groups) from staff and students. Schools were randomly assigned to either the treatment or comparison condition. Treatment schools (four middle schools and four high schools) received training and technical assistance for 2 years to implement the Foundations program. Comparison schools (four middle schools and three high schools) demonstrated the variations in behavioral and instructional practices that naturally exist among schools to provide a comparison for the effect of the MTSS framework.
Randomization occurred at the school level. Before randomization, middle and high schools were placed in strata based on three key school-level variables (e.g., middle vs. high school, baseline School-wide Evaluation Tool (SET) scores, and number of school suspensions per 100 students). Eight middle and seven high schools were assigned at random to treatment or control conditions from within each stratum.
For the MDS3 Student Climate Survey, a representative random sample of English classrooms at each of the 15 treatment and control schools was selected. The classroom selection process took into account representativeness across grades, types of students, and burden on the school to administer the survey.
Longitudinal: Cohort / Event-based
Students and staff at eight middle and seven high schools in one school district located in the Rio Grande Valley in Texas.
In the first year, 947 of 1,932 staff (49 percent) completed the survey; in the second year, 876 of 1,831 staff (48 percent) completed it; and in the third year, 924 of 1,836 staff (50 percent) completed it.
In the first year, 755 of 15,213 students (4.9 percent) completed the survey; in the second year, 1804 of 15,474 students (11.6 percent) completed it; and in the third year, 1708 of 15,443 students (11 percent) completed it.
Modified version of the Maryland Safe and Supportive Schools (MDS3) Climate Survey (Bradshaw et al., 2012)