Impacts and Implementation of the i3-Funded Scale-Up of Success for All (ICPSR 36387)

Version Date: Sep 12, 2016

Cite this study |

Principal Investigator(s):
Janet Quint, MDRC

Series:

https://doi.org/10.3886/ICPSR36387.v1

Version V1

Slide tabs to view more

Alternate Title

Success for All (SFA)

Summary

The Impacts and Implementation of the i3-Funded Scale-Up of Success for All study examined the implementation and impacts of a whole-school reform model, Success for All (SFA), which seeks to increase reading proficiency among students in elementary schools serving low-income children, as this model was scaled up under an Investing in Innovation (i3) grant from the United States Department of Education. The impact evaluation used a cluster random assignment design in which 37 schools in five school districts were selected at random either to implement SFA (the program group schools) or to put in place their business-as-usual reading program (the control group schools). This collection contains Parts 1 to 7 for public use; and Part 8 for restricted use only.

In Part 1: Primary Student Sample data, children who began kindergarten in sampled schools and remained in them through second grade were assessed using standardized assessments of phonics skills, reading fluency, and comprehension. In Part 2: Auxiliary Sample, additional analyses examined third-year impacts on reading skills for all second graders, whether or not they remained continuously enrolled in the study schools, and for students in grades 3 through 5. The analysis also examined impacts for subgroups defined by ethnicity, early literacy skills measured upon entry into kindergarten, and other variables.

Part 3: School Achievement Snapshot Data contain implementation data for analysis using an instrument created by SFA's developers that measures the extent to which program elements were put in place at each program group at the end of each year. This instrument was utilized to assess fidelity to the program model. The implementation analysis was also informed by principal (Part 5: Principal Survey) and teacher surveys (Part 4: Teacher Survey) in all schools, as well by logs (Part 6: Teacher Log) that asked teachers to document their instructional practices in early reading classes. These data, collected in all three study years, make it possible to measure implementation over the study period and to compare program group schools and control group schools on a variety of dimensions.

Part 7: Scale-Up Sample Data provided data on the schools that adopted SFA (and those which were approached, but did not adopt), and schools that adopted before the Investing in Innovations grant. Part 8 is the restricted version of Part 7, which retains geographic location variables.

Citation

Quint, Janet. Impacts and Implementation of the i3-Funded Scale-Up of Success for All. Inter-university Consortium for Political and Social Research [distributor], 2016-09-12. https://doi.org/10.3886/ICPSR36387.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

United States Department of Education

Subject Terms

Geographic Coverage

Smallest Geographic Unit

District (anonymized)

Restrictions

The variables LATCOD and LONCOD are only being made available in Part 8: Scale-Up Sample Data for Restricted Use. Part 8 is for restricted use only.

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2011 -- 2014

Date of Collection

2011 -- 2014

Data Collection Notes

  1. Part 8 is for restricted use only; it contains the same data as Part 7, with geographic location variables included: longitude and latitude.

  2. For more information about the Success for All study, please refer to the MDRC Web site.

Hide

Study Design

MDRC employed a cluster random assignment research design to determine whether Success for All was as effective in its larger-scale version as it had been in earlier tests of the program. In this design, 37 schools located in five districts were assigned at random: 19 to receive the program and the remaining 18 to constitute a "business-as-usual" control group.

The evaluation compared the reading outcomes of kindergartners in both sets of schools in the first year of the study, and of first-graders and second-graders in the next two years, using age-appropriate assessments that were administered individually to each child. During the second and third years of the study, students in third through fifth grades were also assessed, this time in a group setting. Student scores on state standardized tests served as additional measures of reading skills. The impact analysis also measured the program's effects on student attendance and grade promotion.

Impacts were unlikely to occur unless schools were able to implement the program with reasonable fidelity. MDRC collected and analyzed data from a variety of sources -- teacher logs, principal and teacher surveys, and rating sheets completed by the coaches -- to examine the implementation experiences of the schools participating in the study. The analysis also compared the characteristics and implementation levels of the study schools with those of other schools that put Success for All in place as part of the i3 replication.

Sample

37 schools within 5 pre-specified school districts in 4 states agreed to participate in the study. The schools within each district were randomly assigned to program or control conditions, which produced 19 program group schools and 18 control group schools.

Time Method

Longitudinal: Cohort / Event-based

Universe

37 schools including their teachers, principles and students in five school districts in four states that agreed to participate in the study.

Unit(s) of Observation

School, Individual

Data Type(s)

Mode of Data Collection

Response Rates

The response rate varied by instrument ranging from 85% to 100%

Presence of Common Scales

Woodcock-Johnson Letter Word Identifcation, Woodcock-Johnson Word Attack, Woodcock-Johnson Passage Comprehension, Test of Word Reading Efficiency, Gates-MacGinitie Reading Test-Vocabulary, Gates-MacGinitie Reading Test-Comprehension

Hide

Original Release Date

2016-09-12

Version History

2018-02-15 The citation of this study may have changed due to the new version control system that has been implemented. The previous citation was:
  • Quint, Janet. Impacts and Implementation of the i3-Funded Scale-Up of Success for All. ICPSR36387-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2016-09-12. http://doi.org/10.3886/ICPSR36387.v1

2016-09-12 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Checked for undocumented or out-of-range codes.
Hide

Weight

Part 2, Auxiliary Student Sample Data, contains three weight variables: SCH26_GR3_WT, SCH26_GR4_WT, and SCH26_GR5_WT. In general, ICPSR provides unweighted data, but Part 3, School Achievement Snapshot Data, contains weighted variables of scores. Users are highly encouraged to refer to the User Guide for more information on weights.

Hide

Notes