A Process and Outcome Evaluation of the 4-H Mentoring/Youth and Families with Promise (YFP) Program [Utah, 2005-2010] (ICPSR 33781)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Hide

Study Purpose View help for Study Purpose

The purpose of this study was to evaluate the success and effectiveness of the 4-H Mentoring/Youth and Families with Promise (YFP) Program. The degree to which the program was implemented and operated as planned was assessed as well as measuring its impact on at-risk youth, including the long-term outcomes of program participation and the sustainability of those effects.

Study Design View help for Study Design

The process evaluation of Utah's 4-H Mentoring/Youth and Families with Promise (YFP) program assessed the degree to which the program was implemented and operated as planned both as a whole as well as measuring variances across program site locations. The assessment addressed characteristics of the population, mentor-mentee relationships, characteristics of mentors, issues related to length of youth stay, type of program activities, and familial involvement of youth in the program. These were measured using the following tools:

• Site coordinator interviews and dosage data
• Observations of program activities
• Mentor surveys
• Interviews and surveys of program staff
• Youths interviews
• Socio-demographic information
• Program documentation were used to assess program implementation (manuals).

The longitudinal outcome evaluation of the YFP program collected data over three waves:

Wave 1: Pretest -- administered at the start of the program, usually at the beginning of the school year. Collected for YFP youths, parents, and the comparison group (similarly aged youths who had no involvement with the program).

Wave 2: Post-test -- administered when the youth completed the program, or at the end of the school year. If the youth stayed in the program longer than one year, then subsequent post-tests were administered until the youth was no longer in the program (typically 8-32 months from program start). Survey data for the youths in the control group was collected at the end of the school year.

Wave 3: Follow-Up -- telephone interviews were conducted with YFP and comparison youths annually up to three years after program discharge/date of last post-test for treatment group and after last post-test for the comparison group. They were used to assess current academic performance, interpersonal competence, family relationships, and delinquency/risky behavior as well as perceptions of YFP program involvement (in YFP youths only).

The outcome indicators were measured using:

• The Behavioral and Emotional Rating Scale Version 2 (BERS-2)
• Official school and court records
• Goals that youth and parents had about participation in the program
• Socio-demographic and background information
• Youth self-reports of behavior and perceptions of the program

The at-risk youth surveys were typically administered at YFP or FNO events, mailed, or taken during a home visit by staff. The comparison group students took the tests at school.

Sample View help for Sample

The treatment group included youths and their parents served by the YFP program from 2005-2010. The comparison group was a purposive sample of similarly aged school students.

Time Method View help for Time Method

Universe View help for Universe

Youths served by the 4-H Mentoring/Youth and Families with Promise program who attended various school districts in Utah from 2005-2010, and a comparison group of youths similarly aged to program youths was measured by the outcome evaluation. Additionally, the YFP programs themselves served as a the focus of process evaluation.

Unit(s) of Observation View help for Unit(s) of Observation

Data Source View help for Data Source

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Description of Variables View help for Description of Variables

There are 741 cases and 1,184 variables in data set 1 including information from multiple time periods (pre, post, and follow-up surveys) and cohorts. There are various categories of assessment including:

1. The Behavioral and Emotional Rating Scale Version 2 (BERS-2), a 52 item scale, used to create a multitude of variables that measured the behavioral and emotional strengths of children. It was administered during Pre and Post tests and is comprised of five subscales:

   • Interpersonal Strength -- ability to control emotions or behaviors in social situations -- 15 items
   • Intrapersonal Strength -- outlook on competence and accomplishments -- 11 items
   • Affective Strength -- ability to accept affection from and express feelings toward others -- 7 items
   • Family Involvement -- participation in and the relationship with family -- 10 items
   • School Functioning -- competence in school and classroom tasks -- 9 items

2. Career Strength Scale -- This measured interest and aptitude for career and vocational development -- 5 items

3. Youth Mentor Relationship Questionnaire -- 20 item scale with four subscales "not dissatisfied", "helped to cope", "not unhappy" "trust not broken". This was measured on a scale of 1-4 with higher scores corresponding to more successful relationships.

4. Several variables identify delinquent behavior. The Utah Administrative Office of the Courts provided information on each incident which came to the attention of the court for each youth, including the incident date, charges, intake decision, and conviction. Self-reported delinquent behavior was collected in the pre- and post-tests on the number of times the behavior occurred.

5. Socioeconomic and family status questions ranging from whether the youth qualifies for free lunch, who is living with the youth, government services that are being received are also present in the data. Responses are reported through scales and direct wording used by the youth.

6. Date variables -- Date variables were collected for pre- and post-test completion, discharge date, and study discharge date.

7. School Data -- School absences and GPA records.

8. Court Data recording delinquent incidences.

9. Analysis comparing pre- and post-test score changes for BERS-2 scale.

10. Variables considering the matching of youth to mentor, including the reason the match occurred, the date the match occurred, and whether the youth unmatched to the mentor and why.

11. Variables concerning the date the youth and mentor manuals were issued, and the date the parent was given their guide.

12. Variables considering the type of mentor that the youth was given, and whether the mentor attended orientation.

Data Set 2 consists of 91 cases and 104 variables representing survey data from the YFP program mentors. Questions focused on the mentor's length of service in YFP, the matching that took place with the mentee, interactions with their mentee, and their evaluation of mentee success.

Response Rates View help for Response Rates

The response rate varied by data collection point. There were 401 youths in the treatment group and 327 in the comparison group.

Experimental group

• Pre-test: 392
• Post-Test 1 Only (no Pre-Test): 9
• Pre-Test and Post-Test 1: 270
• Pre-Test and Post-Test 2: 98
• Pre-Test, Post-Test 1 and Post-Test 2: 93
• Post-Test 3: 28
• Follow-up interview: year one: 40 percent, year 2: 31 percent, year 3: 30 percent
• School data: 39 percent
• Court data: 82 percent

Comparison Group

• Pre-test: 327
• Pre-Test and Post-Test 1: 280
• Pre-Test and Post-Test 2: 144
• Pre-Test, Post-Test 1 and Post-Test 2: 131
• Follow-up interview: year one: 37 percent, year 2: 49 percent, year 3: 36 percent
• School data: 73 percent
• Court data: 98 percent

Presence of Common Scales View help for Presence of Common Scales

Behavioral and Emotional Rating Scale (BERS) - 2nd edition Youth-Mentor Relationship Questionnaire (YMRQ)

Hide

Original Release Date View help for Original Release Date

2015-12-23

Version History View help for Version History

Hide