youths at risk
Smallest Geographic Unit:
Date of Collection:
Unit of Observation:
Youths served by the 4-H Mentoring/Youth and Families with Promise program who attended various school districts in Utah from 2005-2010, and a comparison group of youths similarly aged to program youths was measured by the outcome evaluation. Additionally, the YFP programs themselves served as a the focus of process evaluation.
administrative records data,
The purpose of this study was to evaluate the success and effectiveness of the 4-H Mentoring/Youth and Families with Promise (YFP) Program. The degree to which the program was implemented and operated as planned was assessed as well as measuring its impact on at-risk youth, including the long-term outcomes of program participation and the sustainability of those effects.
The process evaluation of Utah's 4-H Mentoring/Youth and Families with Promise (YFP) program assessed the degree to which the program was implemented and operated as planned both as a whole as well as measuring variances across program site locations. The assessment addressed characteristics of the population, mentor-mentee relationships, characteristics of mentors, issues related to length of youth stay, type of program activities, and familial involvement of youth in the program. These were measured using the following tools:
- Site coordinator interviews and dosage data
- Observations of program activities
- Mentor surveys
- Interviews and surveys of program staff
- Youths interviews
- Socio-demographic information
- Program documentation were used to assess program implementation (manuals).
The longitudinal outcome evaluation of the YFP program collected data over three waves:
Wave 1: Pretest -- administered at the start of the program, usually at the beginning of the school year. Collected for YFP youths, parents, and the comparison group (similarly aged youths who had no involvement with the program).
Wave 2: Post-test -- administered when the youth completed the program, or at the end of the school year. If the youth stayed in the program longer than one year, then subsequent post-tests were administered until the youth was no longer in the program (typically 8-32 months from program start). Survey data for the youths in the control group was collected at the end of the school year.
Wave 3: Follow-Up -- telephone interviews were conducted with YFP and comparison youths annually up to three years after program discharge/date of last post-test for treatment group and after last post-test for the comparison group. They were used to assess current academic performance, interpersonal competence, family relationships, and delinquency/risky behavior as well as perceptions of YFP program involvement (in YFP youths only).
The outcome indicators were measured using:
- The Behavioral and Emotional Rating Scale Version 2 (BERS-2)
- Official school and court records
- Goals that youth and parents had about participation in the program
- Socio-demographic and background information
- Youth self-reports of behavior and perceptions of the program
The at-risk youth surveys were typically administered at YFP or FNO events, mailed, or taken during a home visit by staff. The comparison group students took the tests at school.
The treatment group included youths and their parents served by the YFP program from 2005-2010. The comparison group was a purposive sample of similarly aged school students.
Longitudinal: Cohort/ Event-based
Mode of Data Collection:
coded on-site observation,
Outcome evaluation: Pre- and post-program self-reported surveys of delinquency behavior were administered to the experimental and comparison youths and parents of experimental youths.
Outcome evaluation: Follow-up surveys up to three years after the YFP program.
Outcome evaluation: YFP mentor surveys
Outcome evaluation: Utah Administrative Office of the Courts records
Outcome evaluation: Official school records
Process evaluation: Site coordinator interviews, surveys and reports about the implementation of YFP at their site
Process evaluation: Observations of program activities
Process evaluation: Interviews with youth
Process evaluation: YFP program guide and mentor manual
Description of Variables:
There are 741 cases and 1,184 variables in data set 1 including information from multiple time periods (pre, post, and follow-up surveys) and cohorts. There are various categories of assessment including:
The Behavioral and Emotional Rating Scale Version 2 (BERS-2), a 52 item scale, used to create a multitude of variables that measured the behavioral and emotional strengths of children.
It was administered during Pre and Post tests and is comprised of five subscales:
- Interpersonal Strength -- ability to control emotions or behaviors in social situations -- 15 items
- Intrapersonal Strength -- outlook on competence and accomplishments -- 11 items
- Affective Strength -- ability to accept affection from and express feelings toward others -- 7 items
- Family Involvement -- participation in and the relationship with family -- 10 items
- School Functioning -- competence in school and classroom tasks -- 9 items
Career Strength Scale -- This measured interest and aptitude for career and vocational development -- 5 items
Youth Mentor Relationship Questionnaire -- 20 item scale with four subscales "not dissatisfied", "helped to cope", "not unhappy" "trust not broken". This was measured on a scale of 1-4 with higher scores corresponding to more successful relationships.
Several variables identify delinquent behavior. The Utah Administrative Office of the Courts provided information on each incident which came to the attention of the court for each youth, including the incident date, charges, intake decision, and conviction. Self-reported delinquent behavior was collected in the pre- and post-tests on the number of times the behavior occurred.
Socioeconomic and family status questions ranging from whether the youth qualifies for free lunch, who is living with the youth, government services that are being received are also present in the data. Responses are reported through scales and direct wording used by the youth.
Date variables -- Date variables were collected for pre- and post-test completion, discharge date, and study discharge date.
School Data -- School absences and GPA records.
Court Data recording delinquent incidences.
Analysis comparing pre- and post-test score changes for BERS-2 scale.
Variables considering the matching of youth to mentor, including the reason the match occurred, the date the match occurred, and whether the youth unmatched to the mentor and why.
Variables concerning the date the youth and mentor manuals were issued, and the date the parent was given their guide.
Variables considering the type of mentor that the youth was given, and whether the mentor attended orientation.
Data Set 2 consists of 91 cases and 104 variables representing survey data from the YFP program mentors. Questions focused on the mentor's length of service in YFP, the matching that took place with the mentee, interactions with their mentee, and their evaluation of mentee success.
The response rate varied by data collection point. There were 401 youths in the treatment group and 327 in the comparison group.
- Pre-test: 392
- Post-Test 1 Only (no Pre-Test): 9
- Pre-Test and Post-Test 1: 270
- Pre-Test and Post-Test 2: 98
- Pre-Test, Post-Test 1 and Post-Test 2: 93
- Post-Test 3: 28
- Follow-up interview: year one: 40 percent, year 2: 31 percent, year 3: 30 percent
- School data: 39 percent
- Court data: 82 percent
- Pre-test: 327
- Pre-Test and Post-Test 1: 280
- Pre-Test and Post-Test 2: 144
- Pre-Test, Post-Test 1 and Post-Test 2: 131
- Follow-up interview: year one: 37 percent, year 2: 49 percent, year 3: 36 percent
- School data: 73 percent
- Court data: 98 percent
Presence of Common Scales:
Behavioral and Emotional Rating Scale (BERS) - 2nd edition
Youth-Mentor Relationship Questionnaire (YMRQ)
Extent of Processing: ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of
disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major
statistical software formats as well as standard codebooks to accompany the data. In addition to
these procedures, ICPSR performed the following processing steps for this data collection:
Created variable labels and/or value labels.
Standardized missing values.
Performed recodes and/or calculated derived variables.
Checked for undocumented or out-of-range codes.