The Bully-Proofing Your School (BPYS) outcome evaluation consisted of school climate surveys administered to elementary school students (Part 1), middle school students (Part 2), and staff (Part 3) in both the treatment and the comparison schools. The design of the data collection for the study was a repeated cross-sectional design. Within each cross-section, the basic design was a hierarchically nested multilevel design. Students were nested within the schools, and schools were split between treatment and comparison conditions, with unequal numbers of cases and potentially different distributions, ranges, and variances on outcome variables.

The evaluation of BPYS took place over five years. In the spring semesters of 2002, 2003, 2004, 2005, and 2006, all participating schools completed a school climate survey. The researchers collected 4,136 completed elementary school surveys (Part 1), 1,627 completed middle school surveys (Part 2), and 1,209 completed staff surveys (Part 3). More specifically, in year one, the 2001-2002 school year, the treatment and comparison schools were identified in the fall, and baseline student and staff outcome evaluation surveys were conducted with the schools during the spring semester. Concurrently, the teachers in the treatment schools were to begin receiving training, but did not yet begin any program implementation. In years two, three, and four, 2002-2005, the treatment schools implemented BPYS and outcome evaluation surveys were completed during the spring semesters in the treatment and comparison schools. Year five, the 2005-2006 school year, was the post-implementation year. In Spring 2006, the school climate survey was conducted again with both staff and students.

For the elementary (Part 1) and middle school (Part 2) students, the mode of data collection was an in-class (group administration) anonymous self-completed survey. The research design required active parental consent for participation in both the elementary and middle school surveys. Active parental consent meant that parents were requested to fill out a form indicating their willingness to have their child participate in the survey. If a parent said no, or if a parent did not return the form, the child was not included in the study. Members of the research team returned to the school several times per week for approximately two weeks to pick up returned forms and to serve as a reminder to the students to bring back their signed consent forms. Once the consent process had begun, the research team scheduled times with the teachers to return to the school and conduct the survey with the students. On the day of the survey, when the researchers first entered the classroom, they consulted with the teacher to be sure only those students with consent remained for the survey. The researcher guided the students in completing their assent forms, which asked the students to write and sign their names and then to indicate whether or not they wanted to participate in the survey. Next, the researcher guided the students through the sociodemographic portion of the surveys and showed the student how to use their colored piece of paper to cover up their answers. At the end, the research team collected the completed surveys and thanked the students for their participation.

For the 1,209 staff surveys (Part 3), the mode of data collection was a mail questionnaire. On the first day of student surveys in a school, the research team placed a packet in the mailbox of each teacher and each school staff member who had any contact with the students, including cafeteria workers and maintenance staff. The packet consisted of a letter explaining the staff survey, a copy of the staff survey, and a postage-paid envelope that allowed the staff member to mail the survey back to the research staff anonymously. Once the data arrived, the surveys were logged in and put into locked file cabinets. They were only removed to enter the data into the computer and were then returned to the locked cabinets as soon as possible thereafter.

Part 1 variables include 8 sociodemographic and general school information items, 23 items relating to school climate, 46 items pertaining to school safety: attitudes and aggressive behavior (perpetration, victimization, and witnessing), and 14 home and family environment variables. Also included are two reverse scored variables and a filter variable which can be used to select the 3,497 cases that were used in the original analyses. More specifically, the eight sociodemographic and general school information variables are case id, year, school id code, type (treatment or comparison), grade, sex, race, and self-reported school performance. The 23 school climate variables asked students how they felt about their school and the people in their school. Of the 46 questions pertaining to school safety, 5 questions asked students about how safe they felt at school, 7 questions asked about their friends, 21 questions asked about things that went on at their school, 6 questions asked about other students at their school, and 7 questions asked about how students felt about hurting other people. The 14 home and family environment questions asked students general questions about their relationship with their parents and family, and about their home life.

Part 2 variables include 8 sociodemographic and general school information variables, 34 variables relating to school climate, 92 variables pertaining to school safety: attitudes and aggressive behavior (perpetration, victimization, and witnessing), 22 variables about substance use, 12 home and family environment variables, 2 variables about guns, 8 variables on activities the respondent participated in, and 6 school attendance variables. Also included are 10 variables that were reverse coded. More specifically, the eight sociodemographic and general school information variables are case id, year, school id code, type (treatment or comparison), grade, sex, race, and self-reported school performance. The 34 school climate questions asked students how they felt about their school and the people in their school. Of the 92 questions pertaining to school safety, 7 questions asked students how safe they felt at school, 14 questions asked about their friends, 58 variables asked about things that went on at their school, including 12 questions about things the students saw in the past month, 24 questions asked about things that the respondent had done at school in the past month, 16 questions asked about things others had done to the respondent in the past month, and 6 questions asked about things that happened at school during the school year. Furthermore, 6 questions asked about bullying and intimidation at school, and 7 questions asked about how students felt about hurting other people. The 22 substance abuse variables asked about drug and alcohol use at school, and the 14 home and family environment questions asked students general questions about their relationship with their parents and family, and about their home life.

Part 3 variables include 7 school and staff characteristics variables, 13 questions about general conditions in the school, 5 questions on how the respondent felt about other people working at the school, 3 questions concerning the resources and participation in the school and the community, 41 questions regarding staff perceptions of safety at the school, and a semester (fall/spring) variable.