Evaluation of Internet Safety Materials Used by Internet Crimes Against Children (ICAC) Task Forces in School and Community Settings, 2011-2012 [United States] (ICPSR 34371)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Data Collection Notes View help for Data Collection Notes

Study Purpose View help for Study Purpose

The purpose of this study was to conduct content and process evaluations of current internet safety education (ISE) program materials and their use by law enforcement presenters and schools.

Study Design View help for Study Design

The study was divided into four subprojects. First, a systematic review or "meta-synthesis" was conducted to identify effective elements of prevention. The coding of the meta-analyses proceeded in two stages. The first stage involved a qualitative review in which a total of four senior project staff identified the program components analyzed in each meta-analysis. Coders were instructed to identify program level variables and participant level variables. The second coding process was conducted using a quantitative coding form derived from the set of components identified in the first review. The process resulted in the development of a KEEP (Known Elements of Effective Prevention) Checklist.

Second, a content analysis (Content Analysis Coding Data) was conducted on four of the most well-developed and long-standing youth Internet safety curricula: i-Safe, iKeepSafe, Netsmartz, and Web Wise Kids. Coders first reviewed all available electronic and written materials for each of the internet safety education (ISE) programs to gather information on the breadth of the program. If a program or lesson focused on digital literacy or was aimed at elementary school-aged youth, the short coding form was used. The short coding form focused on eight popular pre-specified ISE messages. If a curriculum or lesson was directed at middle or high-school youth, and dealt with cyber-bullying, Internet predators, or sexting, the materials were reviewed by coders using the full long-form coding process. This included coding key ISE messages but also included a quantitative coding process to analyze the degree that curricula incorporated educational strategies known to be most effective (KEEP Checklist-Known Elements of Effective Prevention Education); and incorporated research-based messages (Internet Safety Education Fact Checking Sheets.

Third, researchers conducted a process evaluation to better understand how Internet safety education programs are being implemented. The process evaluation was conducted via national surveys with three different groups of respondents: Internet Crimes Against Children (ICAC) Task Force commanders (N=43), ICAC Task Force presenters (ICAC Presenter Survey Data, N=91), and a sample of school professional (School Survey Data, N=139). The ICAC Task Force surveys were completed between February and March 2011. Email invitations were initially sent to Task Force commanders and respondents to the survey were either the commander themselves or someone identified by the commander as most knowledgeable about Task Force ISE efforts. Task Force commanders provided contact information for ICAC presenters. Surveys were emailed to ICAC Task Force presenters in the spring of 2012. The school ISE survey was conducted in June and July of 2012. School respondents were recruited through email and educational listservs.

Finally, researchers developed an Internet safety education outcome survey (Pilot ISE Outcome Survey Data) focused on online harassment and digital citizenship. The anonymous online survey was administered to 1051 students in the 6th through 10th grades at five middles schools and one high school in New Hampshire. Consent and assent procedures were approved by the University of New Hampshire's Institutional Review Board (IRB) and by each participating school principal prior to administration. Parents were sent information about the study two weeks ahead of time and were instructed that they could let the school know if they did not want their youth to participate. Students were told, prior to the beginning, that the survey was anonymous and that they could skip questions or submit a blank survey if they did not want to participate. Survey administration occurred at school computers and was overseen by a contact person at each school.

Sample View help for Sample

For the meta-analysis, Psychinfo, Medline, Criminal Justice Abstracts, ERIC, and the library of Campbell Collaborative were searched comprehensively using multiple keyword variations for summaries, reviews, and meta-analysis of youth prevention program evaluations across different youth problem areas such as drug abuse, sex education, smoking prevention, suicide, youth violence, and school failure. Researchers selected abstracts that met the following definition: "An article, report, or book chapter published between 1990 and 2012 that summarized, contrasted, or compared the effectiveness of two or more prevention programs or approaches delivered to youth and targeting social, emotional or behavioral problems". The search resulted in a total of 424 documents included 73 meta-analyses.

The sample for the content analysis (Content Analysis Coding Data, n= 33), the formal review of materials from the four internet safety education (ISE) programs had to meet the following criteria: materials were directed at youth; materials covered internet safety education; and the materials were accompanied by curriculum or presentation recommendations such as a presenter's or teacher's manual, suggested discussion questions, or activity cards.

For the process evaluation, all 61 ICAC Task Force commanders were invited by email to participate in the survey. Presenter (ICAC Presenter Survey data, n=91) contact information provided by the ICAC Task Force commanders was used to recruit respondents for the ICAC Task Force presenter survey. One hundred and thirty-one ICAC Task Force presents were invited by direct email to participate in the survey. Researchers asked these contacts to forward email invitations with the survey link to individuals who conducted Internet Safety presentations for ICAC Task Forces and posted the recruitment email on the ICAC Task Force listserv and website. An additional 58 respondents completed the survey following these efforts, for a total sample of 111 respondents. Twenty respondents had not presented any ISE in the past year and were eliminated from the sample, resulting in a total sample size of 91 ICAS ISE presenter respondents from 34 states. For the school ISE survey (School Survey Data, n=139), researchers asked each ICAC presenter respondent for the contact information of the person who had coordinated their presentation. Only seventeen ICAC presenters were willing or able to provide the requested information. To increase the sample size, researchers posted emails with a link to the school survey on several educational listservs, including three email forums serving school librarians. The final sample for the school sample included 139 school respondents from 32 states.

The sample for the Internet safety education outcome survey (Pilot ISE Outcome Survey Data, n=1051) include 1,065 students from five middle schools and one high school in New Hampshire between January 17 and April 12, 2012. Survey responses from fourteen youth were eliminated either because there was extensive missing data or because response patterns suggested that it was highly probably they were fake response.

Time Method View help for Time Method

Universe View help for Universe

The universe for the meta-analysis and content analysis includes all internet safety education related materials. The universe for the process evaluation includes all Internet Crimes Against Children (ICAC) Task Force commanders, presenters and school professionals in the United States in 2012. The universe for the outcome evaluation includes all middle school and high school students in New Hampshire between January and April 2012.

Unit(s) of Observation View help for Unit(s) of Observation

Data Source View help for Data Source

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Description of Variables View help for Description of Variables

The content analysis (Content Analysis Coding Data, n=33, 75 variables) includes variables on the program name, the type of materials coded (video, power point slides, interactive web-based programs, or lesson plans), type of content covered (internet predators, sexting, harassment or cyber bullying, privacy or other), targeted age range, number of lessons required, and messages contained in the content, and how the program scored on the KEEP (Known Elements of Effective Prevention) Checklist.

The ICAC Task Force Presenters Survey data (n=91, 173 variables) includes demographic variables on the respondent (sex, age, employment situation, years presenting ISE), setting of last ISE presentation, age of audience, racial make up of the audience, topic of presentation, and type of ISE program used.

The School Survey data (n=139, 126 variables) includes variables on the school type and grades in the school, types of internet safety education (ISE) used, reasons why the school wanted ISE, topics addressed, and type of ISE curricula used.

The Pilot ISE Outcome Survey data (n=1051, 98 variables) includes demographics on the respondent (age, gender, grade, and race), frequency of respondent internet and cell phone use, if the respondent had received any internet safety education, whether they respondent had been the victim of any cyber bullying or harassment and what steps were taken to address the problem, whether the respondent was aware of others being the victim of cyber bullying, and what the respondent might do if they became aware of cyber bullying in the future.

Response Rates View help for Response Rates

Response rates for the meta-analysis and content analysis are not applicable.

For the process evaluation, 43 of 61 ICAC Task Force Commanders completed surveys, for a response rate of 70 percent. The ICAC Task Force Presenter survey was completed by 53 of the 131 presenters recruited through direct email for a response rate of 35 percent. The total sample for the School ISE survey included 139 respondents.

The outcome survey included 1,051 student respondents.

Presence of Common Scales View help for Presence of Common Scales

None.

Original Release Date View help for Original Release Date

2016-03-31

Version History View help for Version History

Weight View help for Weight

None.