Measures of Effective Teaching: 1 - Study Information (ICPSR 34771)

Version Date: Sep 19, 2018

Cite this study |

Principal Investigator(s):
Bill and Melinda Gates Foundation

Series:

https://doi.org/10.3886/ICPSR34771.v3

Version V3 ()

  • V3 [2018-09-19]
  • V2 [2014-12-13] unpublished
Slide tabs to view more

Alternate Title

MET 1 - Study Information

Summary

The Measures of Effective Teaching Project (MET)

The MET project is based on two premises: First, a teacher's evaluation should depend to a significant extent on his/her students' achievement gains; second, any additional components of the evaluation (e.g., classroom observations) should be valid predictors of student achievement gain.

Student achievement was measured in two ways -- through existing state assessments, designed to assess student progress on the state curriculum for accountability purposes, and supplemental assessments, designed to assess higher-order conceptual understanding. The supplemental assessments used were Stanford 9 Open-Ended Reading Assessment in grades 4 through 8, Balanced Assessment in Mathematics (BAM) in grades 4 through 8, and the ACT QualityCore series for Algebra I, English 9, and Biology.

Panoramic digital video of classroom sessions were taken of participating teachers and students, teachers submitted commentary on their lessons (e.g., specifying the learning objective) and then trained raters scored the lesson based on classroom observation protocols using the following five observation protocols:

  • Classroom Assessment Scoring System (CLASS), developed by Robert Pianta, University of Virginia
  • Framework for Teaching, developed by Charlotte Danielson
  • Mathematical Quality of Instruction (MQI), developed by Heather Hill, Harvard University, and Deborah Loewenberg Ball, University of Michigan
  • Protocol for Language Arts Teaching Observations (PLATO), developed by Pam Grossman, Stanford University
  • Quality Science Teaching (QST) Instrument, developed by Raymond Pecheone, Stanford University

A subset of the videos also are being scored using an observational protocol developed by the National Board for Professional Teaching Standards (NBPTS) and using the UTeach Observational Protocol (UTOP), developed by the UTeach Preparation Program.

Close to 3,000 teacher volunteers from across the following six, predominantly urban, school districts participated in the MET project: Charlotte-Mecklenburg Schools, Dallas Independent School District, Denver Public Schools, Hillsborough County Public Schools, Memphis City Schools, and the New York City Department of Education. Participants teach math and English language arts (ELA) in grades 4-8, Algebra I, grade 9 English, and high school biology.

The Study Information Release

Contained in this release are a comprehensive user guide, an observation measures report, a randomization file, a video information file, a subject ID file, a teacher demographics file, a student global ID crosswalk file, and a teacher global ID crosswalk file.

  • User Guide - The user guide provides information on all data releases included in the MET longitudinal database. The report describes: (1) the educational research and policy context for the study; (2) research questions addressed by MET researchers; (3) the core study design and sub-studies; (4) the realized study sample; (5) additional information on study instruments and derived measures; and (6) the data files available for secondary analysis.
  • Observation Measures Report - This report includes a description of the scoring design used for each phase and each year of the video observation measures and final reports of the scoring carried out for each measure.
  • Randomization File - Year two of the MET study included a process for randomizing the classroom to which teachers were assigned. Information found in the randomization file includes district, school, section and student IDs, teacher IDs for the teacher a student was randomly assigned to, the actual teacher the student was recorded as having in May of that year, and the actual teacher the student was recorded as having in October of that year. Also included in the file are variables that indicate the grade of the student, their randomization block and exchange group and section each student was assigned to.
  • Video Information File - This file contains descriptive information about the videos captured for the MET project. Variables found in the file include the ID of the video, as well as IDs for the district, school, teacher and section related to each video, the subject, grade, and year of the video and whether the file is available for users to view.
  • Subject ID Crosswalk - This file contains only ID variables and is included to describe the associations between districts, schools, teachers, sections and students.
  • Teacher Demographics File - This file contains descriptive information about the teachers in the MET project. Variables found in the file include the ID of the teacher, as well as IDs for the district, gender, race, years experience, and level of education.
  • Student Global ID Crosswalk File - Students were given new and unique IDs in each year of data collection. This file matches those Student IDs across the 2008-09, 2009-10, 2010-11, 2011-12, 2012-13, and 2013-14 academic years.
  • Teacher Global ID Crosswalk File - Teachers were given new and unique IDs in each year of data collection. This file matchers those Teachers IDs across the 2008-09, 2009-10, 2010-11, 2011-12, 2012-13, and 2013-14 academic years.

Citation

Bill and Melinda Gates Foundation. Measures of Effective Teaching: 1 - Study Information. Inter-university Consortium for Political and Social Research [distributor], 2018-09-19. https://doi.org/10.3886/ICPSR34771.v3

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

Bill and Melinda Gates Foundation

Subject Terms

Geographic Coverage

Smallest Geographic Unit

School District

Restrictions

The Measures of Effective Teaching Longitudinal Database (MET LDB) is restricted from general dissemination; a Confidential Data Use Agreement must be established prior to access. Researchers interested in gaining access to the data can submit their applications via ICPSR's online Restricted Contracting System accessible via the "Access Restricted Data" tab on the ICPSR study homepage.

Applicants will be required to:

  • Submit IRB approval/exemption documentation;
  • Scan and email the completed Confidential Data Use Agreement, signed by the Primary Investigator and an Institutional Representative;
  • Pay annual access fee and renew yearly for continued data access.

Please visit the MET LDB Web site for more information.

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2009 -- 2014

Date of Collection

2009 -- 2014

Data Collection Notes

  1. Participating academic institutions include Dartmouth College, Harvard University, Stanford University, University of Chicago, University of Michigan, University of Virginia, and University of Washington. Participating non-profit organizations include Educational Testing Service, RAND Corporation, and the New Teacher Center. Participating education consultants include Cambridge Education, Teachscape, and Westat. The National Board for Professional Teaching Standards and Teach For America supported the project and have encouraged their members to participate. The American Federation of Teachers and the National Education Association were involved in discussions about the MET project and supported the research.

  2. Users should note that all files with a suffix of "_v1" are new files available through the current update.

  3. For additional information about The Measures of Effective Teaching (MET) Project, please visit the ICPSR MET LDB Web site, as well as the MET Project Data Web site hosted by the Bill and Melinda Gates Foundation.
Hide

Study Purpose

The MET Study addressed several related research questions: How reliable and valid are the specific measures of teaching effectiveness under study? Do the various measures identify distinctive dimensions of teaching effectiveness, and if so, what dimensions are identified? What measures of effective teaching are empirically related to student learning gains? What does effective teaching look like, and how does it compare to less effective teaching? For example, what is the distribution of teacher scores on measures of effective teaching, and how much difference is there in teacher knowledge scores, teaching practice scores, and student outcome scores among teachers at different points in the distribution of measures of effective teaching? Can multiple sources of data on teachers and their teaching be combined to develop a set of fair, valid, and reliable indicators of teaching quality for use in teacher evaluation systems intended to rank teachers for personnel decision making and to promote teachers' professional learning and development?

Study Design

Teachers at targeted grade levels (4th-9th grade) and in targeted subject areas (English Language Arts, Math and Biology) were recruited from schools in six school districts. Video was captured of these teachers' classroom sessions and these videos were scored using multiple measures to rate teaching method and classroom environment. Multiple assessment instruments and standardized test scores were used to gather student achievement data. Multiple surveys were administered to gather student, teacher and principal opinion on topics such as classroom instruction as well as classroom, school and working environment. Year Two of the study included a teacher randomization.

Sample

The MET Study began with a process of "opportunity" sampling that took place during July - November 2009 and resulted in six, large school districts volunteering to participate in the study. The process of opportunistic sampling then continued as elementary, middle, and high schools within each district were recruited into the study. Once schools were recruited, opportunity sampling continued as teachers (at targeted grade levels and subject areas) within these schools volunteered for the study. The sampling process resulted in 2,741 teachers from 317 schools in six large school districts being recruited into the first year of the study. Attrition in the teacher sample in Year Two of the study resulted from schools that dropped out of the study (11 schools; 60 teachers). Additionally, individual teachers dropped out when they left their school or district, began teaching a different subject or grade, lost interest in the study, or became ill. Overall, the Year Two sample of teachers included 2,086 teachers in 310 schools. Of the 582 4th and 5th grade teachers in Year Two, the majority continued to be subject-matter generalists who taught English Language Arts (ELA) and Mathematics to a single class of students, although the sample also included a small number of subject matter specialists (who taught ELA or Mathematics to more than one class section of students) and teachers who volunteered only to have their teaching of a single subject be studied. Of the 841 middle-school grade teachers in Year Two, about half continued to be teachers of ELA in grades 6-8, and the other half teachers of Mathematics at these grades. Of the 479 9th grade teachers in Year Two, about a third were teachers of 9th grade English, another third were teachers of 9th grade Algebra I, and another third were teachers of 9th grade Biology.

Time Method

Longitudinal: Panel

Universe

Teachers and students within the six participating school districts.

Unit(s) of Observation

Teachers, Students

Data Source

Administrative data were gathered from each of the six participating school districts.

Data Type(s)

Mode of Data Collection

Response Rates

2,746 teachers began Year 1 of the MET project and 1,868 completed Year 2 of the MET project.

Hide

Original Release Date

2013-07-26

Version History

2018-09-19 Dataset 1, Randomization File has been updated to link more teachers via the Global IDs, Dataset 3, Subject ID Crosswalk File has been updated to add Global IDs for students and teachers, Dataset 5, Student Global ID Crosswalk File (formerly Multi-Year Student ID Crosswalk File) has been updated to include pairs of ICPSR IDs and Global IDs. Dataset 6, Teacher Global ID Crosswalk File has been added to include the full ICPSR package suite (SPSS, SAS, Stata, R). Released additional documentation includes, a MET overview infographic, a file organization description, and a study organization description, as well as an updated User Guide.

2018-02-15 The citation of this study may have changed due to the new version control system that has been implemented. The previous citation was:
  • Bill and Melinda Gates Foundation. Measures of Effective Teaching: 1 - Study Information. ICPSR34771-v3. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2018-09-19. http://doi.org/10.3886/ICPSR34771.v3

2014-12-13 Update to install a versioning system in the study file names and an update to the User Guide.

2014-07-29 Added a multi-year student ID crosswalk to allow users to match students across academic years.

2014-05-30 Updated Video Information file to resolve issue with Year 1 Biology lab videos listed as unavailable.

2014-04-21 Updating Video Information file to resolve value conflict with survey control file.

2014-02-28 This update included revisions to the User Guide and the addition of the Observational Measures Report.

2013-10-15 The Teacher Demographics file was updated.

2013-09-23 Changed study title and updated documents.

2013-09-16 The Teacher Demographics file was added.

2013-09-16 User Guide was released.

2013-07-26 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Performed consistency checks.
  • Created variable labels and/or value labels.
  • Checked for undocumented or out-of-range codes.
Hide

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.