Evaluation of an Expansion Strategy for the Assessment-to-Instruction Professional Support System, United States, 2018-2021 (ICPSR 38934)

Version Date: Jan 8, 2024

Cite this study |

Principal Investigator(s):
Pei Zhu, MDRC

Series:

https://doi.org/10.3886/ICPSR38934.v1

Version V1

Slide tabs to view more

Summary

The study's primary goal is to assess whether a mixed-mode professional development (PD) model that combines virtual and in-person delivery of PD activities can serve as a viable alternative to the original face-to-face delivery approach in supporting teachers' use of the Assessment-to-Instruction (A2i) system, facilitating their implementation of differentiated small-group instruction, and improving students' reading skills in a large, diverse set of schools.

There are three datasets for this study:

  • A school-level dataset that includes information collected about the 59 schools in the study. School-level information was collected from the 2017-2018 Common Core of Data provided by the National Center of Education Statistics, the 2017-2018 Civil Rights Data Collection provided by the Office of Civil Rights, and school-level reading achievement data during the 2017-2018 school year, provided by state education departments. To maintain anonymity of schools, we excluded any publicly available data about school characteristics from the dataset. In Table 3 in the study report, which compares the study schools to public elementary schools nationally, the national sample includes all public, regular elementary schools serving students in any of Grades K, 1, 2, or 3 that are not charter, magnet or virtual schools.
  • A teacher-level dataset that includes teachers in the analysis sample (i.e. teachers in Grades K-1 in the 2017-2018 school year, expanding to Grades K-3 in the 2020-2021 school year). This dataset includes responses to the two teacher surveys administered in 2019 and 2021, information about teacher usage of the A2i platform, and the number of PD events attended by each teacher in 2019-2020 and 2020-2021.
  • A student-level dataset that includes all information used for the student-level analysis, including student demographic from district records and student achievement data from A2i assessments, Gates-MacGinitie Reading Tests administered in the spring of 2019, and state/district reading tests administered in the spring of 2021.

Citation

Zhu, Pei. Evaluation of an Expansion Strategy for the Assessment-to-Instruction Professional Support System, United States, 2018-2021. Inter-university Consortium for Political and Social Research [distributor], 2024-01-08. https://doi.org/10.3886/ICPSR38934.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

United States Department of Education (U411A170021)

Subject Terms

Geographic Coverage

Restrictions

This data collection may not be used for any purpose other than statistical reporting and analysis. Use of these data to learn the identity of any person or establishment is prohibited. To protect respondent privacy, some of the data files in this collection are restricted from general dissemination. To obtain these restricted files researchers must agree to the terms and conditions of a Restricted Data Use Agreement in accordance with existing ICPSR servicing policies.

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2018-06-01 -- 2021-06-01

Date of Collection

2018-06-01 -- 2021-06-01 (Fall 2018 through Spring 2021)

Data Collection Notes

  1. Each observation in the datasets contains RUFSCHID, RUFRABLOCK, and RACODE. RUFSCHID is an anonymized identifier of the study school. RUFRABLOCK is an indicator of the observation's random assignment block. RACODE indicates the random assignment condition for each observation, with a value of 1 for the mixed-mode approach and a value of 0 for the face-to-face approach

  2. For additional information on this study, please visit the MDRC Website .
Hide

Study Purpose

The study's primary goal is to assess whether a mixed-mode professional development (PD) model that combines virtual and in-person delivery of PD activities can serve as a viable alternative to the original face-to-face delivery approach in supporting teachers' use of the A2i system, facilitating their implementation of differentiated small-group instruction, and improving students' reading skills in a large, diverse set of schools.

Study Design

This study recruited 59 elementary schools from 20 districts around the country and randomly assigned roughly equal numbers of schools to either the mixed-mode or the face-to-face PD model within districts or district groups. Early grade teachers in the mixed-mode schools received a combination of virtual and in-person training and support for the A2i Professional Support System, while those in the face-to-face schools received such training and support exclusively in person. This design determines that any differences in teacher experiences or student outcomes between these two groups of otherwise similar schools can be attributed to the different PD delivery models to which they were exposed. Therefore, if the two PD models were fully implemented, the difference in outcomes between these two groups of schools can be considered the differential impact between these two PD models. The study collected quantitative data on schools, teachers, and students to assess whether and to what degree the two PD models affected the teachers and students differently.

Sample

The study recruited 59 elementary schools from 20 school districts in 4 states to participate in this evaluation and, within each district or district group, randomly assigned about half of them to receive PD through the face-to-face model (the face-to-face schools) and the other half through the mixed-mode model (the mixed-mode schools).

Time Method

Longitudinal: Cohort / Event-based

Universe

Students and teachers grades K-3 at participating schools

Unit(s) of Observation

individual, organization

Data Type(s)

Mode of Data Collection

Description of Variables

Prefixes in variable names indicate the year and source of data collection. A prefix of "Y1" indicates data collected during the 2018-2019 school year, "Y2" indicates the 2019-2020 school year, and "Y3" indicates the 2020-2021 school year.

In the teacher-level dataset, each observation contains a unique RUFTCHID. Variable names contain prefixes to indicate the source of each variable, with:

  • TS: teacher survey
  • TA: teacher PD attendance data. For the Y1 school year (2018-2019), where teacher-level data were not available, school-by-grade counts of PD events (also included as Y1COUNT:) were used.
  • TCH: teacher A2i platform usage, provided by A2i platform.

In the student-level dataset, each observation contains a unique RUFSTUID. Variable names contain prefixes to indicate the source of each variable, with:

  • DD: District records. Information used for baseline covariates has a "BLDD" prefix
  • SA: A2i assessment
  • DT: State/district reading test
  • GM: Gates-MacGinitie Reading Test

The prefixes described above combine to describe the data source and date of data collection. For example, a prefix of "Y1TS" indicates data collected as part of the Year 1 (2018-2019) teacher survey. A codebook of all variables has been provided for each file.

Hide

Original Release Date

2024-01-08

Hide

Weight

To obtain the overall estimated differential impact between the two program conditions, estimated impacts for each random-assignment block were pooled by weighting each block by the number of mixed-mode schools in the block.

Hide

Notes