An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012 (ICPSR 36531)

Version Date: Mar 9, 2017

Cite this study |

Principal Investigator(s):
Marie-Andree Somers, MDRC; Jean Grossman, Princeton University

Series:

https://doi.org/10.3886/ICPSR36531.v1

Version V1

Slide tabs to view more

Summary

An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012, funded by the Edna McConnell Clark Foundation's Social Innovation Fund, was conducted in the Summer of 2012 in three school districts (District A, B, C) that were new partnerships for the Building Educated Leaders for Life program (BELL). Of the three study districts, District A and B offered the BELL program in one middle school each; District C offered the program in three schools.

Several types of quantitative data were collected about students and BELL teachers in the study. The nature and purpose of these data sources are described below.

(1) Spring (baseline) characteristics and test scores:During the application process, parents provided information about their child's socioeconomic characteristics (racial or ethnic group, parents' education, and so on. In addition, schools provided information about whether students in the study were eligible for free or reduced-price lunch, whether they had an individualized education plan (IEP), and whether English was their second language. Schools also provided students' scores on the spring 2012 math and reading assessments administered by their state; these scores were used to determine whether students were proficient, based on local cutoff scores on their state test.

(2) Attendance records: BELL provided the evaluation team with the attendance records of students in the study during summer 2012, including any students in the non-BELL group who may have attended the program.

(3) Fall reading and math tests: Students in the study were encouraged to take standardized tests in math and reading in fall 2012. In the average study district, students took the tests six weeks after the end of the program, or one week after the start of the school year. Students' reading achievement was assessed using the Group Reading Assessment and Diagnostic Examination (GRADE), and their math achievement was assessed with its math counterpart, the Group Mathematics Assessment and Diagnostic Examination (GMADE). The GRADE contains two subtests (reading comprehension and vocabulary), and the GMADE contains three (concepts, operations, and processes).

(4) Fall student survey: In the same session as the GRADE and GMADE were administered, students also completed a short survey asking about the extent to which they were engaged in various aspects of instruction when they returned to school in the fall (for example, whether they paid attention in class and whether they completed their homework on time). The survey also included a set of items asking students to describe their activities during the summer.

(5) BELL teacher survey: BELL administers a survey to its teachers as part of regular program monitoring and evaluation activities. The survey captures teachers' characteristics as well as their perceptions of various aspects of the BELL program (such as training, materials, and staffing), of their own performance in the classroom, and of their students' performance and engagement. BELL administers the survey to all of its teachers -- academic (English Language Arts [ELA] or math or both) and enrichment teachers -- as well as mentors (teaching assistants). However, given the academic focus of this evaluation, the target population for the study's purposes was academic teachers who taught students in the study.

This collection contains two datasets. Part 1 is a student-level dataset that includes all information collected about students in the study sample (baseline data, GRADE and GMADE test scores, student survey data, and attendance data). Part 2 is a teacher survey dataset that includes teachers in the analysis sample (i.e. academic middle school teachers who responded to the survey and who taught students in the study).

Demographic information about the students include socioeconomic characteristics like racial or ethnic group, parents' education, and so on. Demographic information about the teachers in the sample include their education and experience backgrounds.

Citation

Somers, Marie-Andree, and Grossman, Jean. An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012. Inter-university Consortium for Political and Social Research [distributor], 2017-03-09. https://doi.org/10.3886/ICPSR36531.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

Social Innovation Fund (Edna McConnell Clark Foundation & Corporation for National and Community Service)

Subject Terms

Geographic Coverage

Smallest Geographic Unit

School district (anonymized)

Restrictions

This data collection may not be used for any purpose other than statistical reporting and analysis. Use of these data to learn the identity of any person or establishment is prohibited. To protect respondent privacy, this collection is restricted from general dissemination. To obtain these files, researchers must agree to the terms and conditions of a Restricted Data Use Agreement in accordance with existing ICPSR servicing policies.

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2012

Date of Collection

2012

Data Collection Notes

  1. The student and teacher data files can be merged using the BELLCLASSROOM variable, which is an identifier for the students' and teachers' classroom. More information can be found within the Description of Data Files Documentation.

  2. For additional information on "An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012", please visit the BELL Web site.
Hide

Study Purpose

The purpose of this study was to evaluate the impact of BELL's summer program for middle school students who are performing below grade level. Also, the results can be useful for generating suggestive or preliminary evidence about the potential effects of a full day, academically oriented summer program model for middle school students.

Study Design

BELL aims to serve students who are struggling academically, and so eligibility for the study was limited to students in three study districts who were performing below grade level. The evaluation of BELL's middle school program uses a student-level random assignment research design to examine BELL's effects on student outcomes. Some of the districts that partner with BELL operate voluntary summer programs where there are more eligible students than BELL has the capacity to serve. In these oversubscribed voluntary programs, random assignment was used to determine which students would be invited to attend the BELL middle school program (the BELL group) and which students would participate in "business as usual" summer activities (the non-BELL group).

In summer 2012, three of the ten districts that partnered with BELL to serve middle school students had oversubscribed voluntary programs and were willing to participate in the evaluation. Of the three study districts, District A and B offered the BELL program in one middle school each; District C offered the program in three schools. The schools in Districts A and C served only rising seventh- and eighth-grade students, whereas the middle school in District B served students in all three middle school grades. In order to make random assignment possible, a further requirement was that students had to be attending the program voluntarily (as opposed to being required to attend because they had not met some criteria on the state test). To be included in the study, students and their parents also had to complete the BELL application form and sign the informed consent form. In total, 1,032 rising sixth-, seventh-, or eighth-grade students applied to the middle school program in the three study districts and agreed to be part of the study. Of these 1,032 students, 385 students are from District A; 127 students are from District B; and the remaining half (520 students) are from District C.

Random assignment was then used to determine which of these students would be invited to participate in the BELL middle school program (the BELL group) and which students would not be invited to participate in BELL (the non-BELL group). In order to ensure that each grade-level classroom in the BELL study sites would have 20 students, the research team conducted a separate random assignment lottery for each grade level, as well as for each school that students attended in the spring before the summer program. In total, 643 students (62 percent of study participants) were randomly assigned to the BELL group, while the remaining 389 students (38 percent of study participants) were placed in the non-BELL group.

Sample

Convenience sample of schools (program sites) and students in the program (students who consented to study participation).

Time Method

Cross-sectional

Universe

Middle school students served by the BELL program in Summer 2012 in a subset of program sites.

Unit(s) of Observation

Individual

Data Type(s)

Mode of Data Collection

Presence of Common Scales

GRADE test, GMADE test, and Skinner student engagement scale.

Hide

Original Release Date

2017-03-09

Version History

2018-02-15 The citation of this study may have changed due to the new version control system that has been implemented. The previous citation was:
  • Somers, Marie-Andree, and Jean Grossman. An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012. ICPSR36531-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2017-03-09. http://doi.org/10.3886/ICPSR36531.v1

2017-03-09 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Checked for undocumented or out-of-range codes.
Hide

Weight

No weight variables are present in the data collection.

Hide

Notes