ICPSR Logo

This study is provided by ICPSR. ICPSR provides leadership and training in data access, curation, and methods of analysis for a diverse and expanding social science research community.

An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012 (ICPSR 36531)

Principal Investigator(s): Somers, Marie-Andree, MDRC; Grossman, Jean, Princeton University

Summary:

An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012, funded by the Edna McConnell Clark Foundation's Social Innovation Fund, was conducted in the Summer of 2012 in three school districts (District A, B, C) that were new partnerships for the Building Educated Leaders for Life program (BELL). Of the three study districts, District A and B offered the BELL program in one middle school each; District C offered the program in three schools.

Several types of quantitative data were collected about students and BELL teachers in the study. The nature and purpose of these data sources are described below.

(1) Spring (baseline) characteristics and test scores:During the application process, parents provided information about their child's socioeconomic characteristics (racial or ethnic group, parents' education, and so on. In addition, schools provided information about whether students in the study were eligible for free or reduced-price lunch, whether they had an individualized education plan (IEP), and whether English was their second language. Schools also provided students' scores on the spring 2012 math and reading assessments administered by their state; these scores were used to determine whether students were proficient, based on local cutoff scores on their state test.

(2) Attendance records: BELL provided the evaluation team with the attendance records of students in the study during summer 2012, including any students in the non-BELL group who may have attended the program.

(3) Fall reading and math tests: Students in the study were encouraged to take standardized tests in math and reading in fall 2012. In the average study district, students took the tests six weeks after the end of the program, or one week after the start of the school year. Students' reading achievement was assessed using the Group Reading Assessment and Diagnostic Examination (GRADE), and their math achievement was assessed with its math counterpart, the Group Mathematics Assessment and Diagnostic Examination (GMADE). The GRADE contains two subtests (reading comprehension and vocabulary), and the GMADE contains three (concepts, operations, and processes).

(4) Fall student survey: In the same session as the GRADE and GMADE were administered, students also completed a short survey asking about the extent to which they were engaged in various aspects of instruction when they returned to school in the fall (for example, whether they paid attention in class and whether they completed their homework on time). The survey also included a set of items asking students to describe their activities during the summer.

(5) BELL teacher survey: BELL administers a survey to its teachers as part of regular program monitoring and evaluation activities. The survey captures teachers' characteristics as well as their perceptions of various aspects of the BELL program (such as training, materials, and staffing), of their own performance in the classroom, and of their students' performance and engagement. BELL administers the survey to all of its teachers -- academic (English Language Arts [ELA] or math or both) and enrichment teachers -- as well as mentors (teaching assistants). However, given the academic focus of this evaluation, the target population for the study's purposes was academic teachers who taught students in the study.

This collection contains two datasets. Part 1 is a student-level dataset that includes all information collected about students in the study sample (baseline data, GRADE and GMADE test scores, student survey data, and attendance data). Part 2 is a teacher survey dataset that includes teachers in the analysis sample (i.e. academic middle school teachers who responded to the survey and who taught students in the study).

Demographic information about the students include socioeconomic characteristics like racial or ethnic group, parents' education, and so on. Demographic information about the teachers in the sample include their education and experience backgrounds.

Access Notes

  • Data in this collection are available only to users at ICPSR member institutions. Please log in so we can determine if you are with a member institution and have access to these data files.

    One or more files in this collection have special restrictions ; consult the restrictions note to learn more.

    This data collection may not be used for any purpose other than statistical reporting and analysis. Use of these data to learn the identity of any person or establishment is prohibited. To protect respondent privacy, this collection is restricted from general dissemination. To obtain these files, researchers must agree to the terms and conditions of a Restricted Data Use Agreement in accordance with existing ICPSR servicing policies.

Dataset(s)

DS0:  Study-Level Files
DS1:  Student-Level Data File (Restricted-use)
Download:
No downloadable data files available.
DS2:  Teacher Survey Data File (Restricted-use)
Documentation:
Download:
No downloadable data files available.

Study Description

Citation

Somers, Marie-Andree, and Jean Grossman. An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012. ICPSR36531-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2017-03-09. https://doi.org/10.3886/ICPSR36531.v1

Persistent URL: https://doi.org/10.3886/ICPSR36531.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote XML (EndNote X4.0.1 or higher)

Funding

This study was funded by:

  • Social Innovation Fund (Edna McConnell Clark Foundation & Corporation for National and Community Service)

Scope of Study

Subject Terms:    academic achievement, education, mathematics, middle schools, programs, reading, students, teachers

Smallest Geographic Unit:    School district (anonymized)

Geographic Coverage:    United States

Time Period:   

  • 2012

Date of Collection:   

  • 2012

Unit of Observation:    Individual

Universe:    Middle school students served by the BELL program in Summer 2012 in a subset of program sites.

Data Type(s):    administrative records data, event/transaction data, experimental data, survey data

Data Collection Notes:

The student and teacher data files can be merged using the BELLCLASSROOM variable, which is an identifier for the students' and teachers' classroom. More information can be found within the Description of Data Files Documentation.

For additional information on "An Analysis of the Effects of an Academic Summer Program for Middle School Students, 2012", please visit the BELL Web site.

Methodology

Study Purpose:    The purpose of this study was to evaluate the impact of BELL's summer program for middle school students who are performing below grade level. Also, the results can be useful for generating suggestive or preliminary evidence about the potential effects of a full day, academically oriented summer program model for middle school students.

Study Design:   

BELL aims to serve students who are struggling academically, and so eligibility for the study was limited to students in three study districts who were performing below grade level. The evaluation of BELL's middle school program uses a student-level random assignment research design to examine BELL's effects on student outcomes. Some of the districts that partner with BELL operate voluntary summer programs where there are more eligible students than BELL has the capacity to serve. In these oversubscribed voluntary programs, random assignment was used to determine which students would be invited to attend the BELL middle school program (the BELL group) and which students would participate in "business as usual" summer activities (the non-BELL group).

In summer 2012, three of the ten districts that partnered with BELL to serve middle school students had oversubscribed voluntary programs and were willing to participate in the evaluation. Of the three study districts, District A and B offered the BELL program in one middle school each; District C offered the program in three schools. The schools in Districts A and C served only rising seventh- and eighth-grade students, whereas the middle school in District B served students in all three middle school grades. In order to make random assignment possible, a further requirement was that students had to be attending the program voluntarily (as opposed to being required to attend because they had not met some criteria on the state test). To be included in the study, students and their parents also had to complete the BELL application form and sign the informed consent form. In total, 1,032 rising sixth-, seventh-, or eighth-grade students applied to the middle school program in the three study districts and agreed to be part of the study. Of these 1,032 students, 385 students are from District A; 127 students are from District B; and the remaining half (520 students) are from District C.

Random assignment was then used to determine which of these students would be invited to participate in the BELL middle school program (the BELL group) and which students would not be invited to participate in BELL (the non-BELL group). In order to ensure that each grade-level classroom in the BELL study sites would have 20 students, the research team conducted a separate random assignment lottery for each grade level, as well as for each school that students attended in the spring before the summer program. In total, 643 students (62 percent of study participants) were randomly assigned to the BELL group, while the remaining 389 students (38 percent of study participants) were placed in the non-BELL group.

Sample:    Convenience sample of schools (program sites) and students in the program (students who consented to study participation).

Time Method:    Cross-sectional

Weight:    No weight variables are present in the data collection.

Mode of Data Collection:    record abstracts, cognitive assessment test, on-site questionnaire, web-based survey

Presence of Common Scales:    GRADE test, GMADE test, and Skinner student engagement scale.

Extent of Processing:   ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Checked for undocumented or out-of-range codes.

Version(s)

Original ICPSR Release:   2017-03-09

Utilities

Metadata Exports

If you're looking for collection-level metadata rather than an individual metadata record, please visit our Metadata Records page.

Download Statistics