A Process and Outcome Evaluation of the 4-H Mentoring/Youth and Families with Promise (YFP) Program [Utah, 2005-2010] (ICPSR 33781)
Version Date: Jan 13, 2016 View help for published
Principal Investigator(s): View help for Principal Investigator(s)
Stan Orchowsky, Justice Research and Statistics Association;
Mary Poulin, Justice Research and Statistics Association
https://doi.org/10.3886/ICPSR33781.v1
Version V1
Summary View help for Summary
This study was an evaluation of Utah's 4-H Mentoring/Youth and Families with Promise (YFP) program, a statewide 4-H mentoring program designed for at-risk youths. Both the degree to which the YFP program was implemented and operated as planned and the long term outcome of program participation and sustainability of program effects were assessed.
The process evaluation addressed (1) characteristics of the population served (risk factors, protective factors, socio-demographic characteristics); (2) type and dosage of program activities; (3) mentor-mentee relationships; (4) satisfaction with the program; (5) involvement of youths' family members in the program; (6) characteristics of mentors (e.g., type of mentor, sex, race); and (7) issues related to the youths' length of stay.
The outcome evaluation consisted of an experimental group of at-risk youth 10-14 years old who were paired with a mentor, participated in 4-H activities, and also attended Family Night Out (FNO) activities to strengthen family bonds. A comparison group was constructed of students of the same age attending the same schools but who did not participate in the YFP program. The outcome evaluation measured whether the YFP program increased youths' interpersonal competence, improved youths' academic performance, strengthened youths' family relationships, and prevented delinquency. Outcome indicators were measured at the start of, during, and up to three years after program involvement. In addition to self-reporting pre/post tests, official school and court records were recorded. Additionally, youths' perceptions of mentor relationship quality were measured with the Youth-Mentor Relationship Questionnaire.
Citation View help for Citation
Export Citation:
Funding View help for Funding
Subject Terms View help for Subject Terms
Geographic Coverage View help for Geographic Coverage
Smallest Geographic Unit View help for Smallest Geographic Unit
program site
Restrictions View help for Restrictions
Access to these data is restricted. Users interested in obtaining these data must complete a Restricted Data Use Agreement, specify the reasons for the request, and obtain IRB approval or notice of exemption for their research
Distributor(s) View help for Distributor(s)
Time Period(s) View help for Time Period(s)
Date of Collection View help for Date of Collection
Study Purpose View help for Study Purpose
The purpose of this study was to evaluate the success and effectiveness of the 4-H Mentoring/Youth and Families with Promise (YFP) Program. The degree to which the program was implemented and operated as planned was assessed as well as measuring its impact on at-risk youth, including the long-term outcomes of program participation and the sustainability of those effects.
Study Design View help for Study Design
The process evaluation of Utah's 4-H Mentoring/Youth and Families with Promise (YFP) program assessed the degree to which the program was implemented and operated as planned both as a whole as well as measuring variances across program site locations. The assessment addressed characteristics of the population, mentor-mentee relationships, characteristics of mentors, issues related to length of youth stay, type of program activities, and familial involvement of youth in the program. These were measured using the following tools:
- Site coordinator interviews and dosage data
- Observations of program activities
- Mentor surveys
- Interviews and surveys of program staff
- Youths interviews
- Socio-demographic information
- Program documentation were used to assess program implementation (manuals).
The longitudinal outcome evaluation of the YFP program collected data over three waves:
Wave 1: Pretest -- administered at the start of the program, usually at the beginning of the school year. Collected for YFP youths, parents, and the comparison group (similarly aged youths who had no involvement with the program).
Wave 2: Post-test -- administered when the youth completed the program, or at the end of the school year. If the youth stayed in the program longer than one year, then subsequent post-tests were administered until the youth was no longer in the program (typically 8-32 months from program start). Survey data for the youths in the control group was collected at the end of the school year.
Wave 3: Follow-Up -- telephone interviews were conducted with YFP and comparison youths annually up to three years after program discharge/date of last post-test for treatment group and after last post-test for the comparison group. They were used to assess current academic performance, interpersonal competence, family relationships, and delinquency/risky behavior as well as perceptions of YFP program involvement (in YFP youths only).
The outcome indicators were measured using:
- The Behavioral and Emotional Rating Scale Version 2 (BERS-2)
- Official school and court records
- Goals that youth and parents had about participation in the program
- Socio-demographic and background information
- Youth self-reports of behavior and perceptions of the program
The at-risk youth surveys were typically administered at YFP or FNO events, mailed, or taken during a home visit by staff. The comparison group students took the tests at school.
Sample View help for Sample
The treatment group included youths and their parents served by the YFP program from 2005-2010. The comparison group was a purposive sample of similarly aged school students.
Time Method View help for Time Method
Universe View help for Universe
Youths served by the 4-H Mentoring/Youth and Families with Promise program who attended various school districts in Utah from 2005-2010, and a comparison group of youths similarly aged to program youths was measured by the outcome evaluation. Additionally, the YFP programs themselves served as a the focus of process evaluation.
Unit(s) of Observation View help for Unit(s) of Observation
Data Source View help for Data Source
Process evaluation: Interviews with youth
Outcome evaluation: Official school records
Outcome evaluation: Utah Administrative Office of the Courts records
Outcome evaluation: Pre- and post-program self-reported surveys of delinquency behavior were administered to the experimental and comparison youths and parents of experimental youths.
Process evaluation: YFP program guide and mentor manual
Process evaluation: Observations of program activities
Outcome evaluation: Follow-up surveys up to three years after the YFP program.
Process evaluation: Site coordinator interviews, surveys and reports about the implementation of YFP at their site
Outcome evaluation: YFP mentor surveys
Data Type(s) View help for Data Type(s)
Mode of Data Collection View help for Mode of Data Collection
Description of Variables View help for Description of Variables
There are 741 cases and 1,184 variables in data set 1 including information from multiple time periods (pre, post, and follow-up surveys) and cohorts. There are various categories of assessment including:
The Behavioral and Emotional Rating Scale Version 2 (BERS-2), a 52 item scale, used to create a multitude of variables that measured the behavioral and emotional strengths of children. It was administered during Pre and Post tests and is comprised of five subscales:
- Interpersonal Strength -- ability to control emotions or behaviors in social situations -- 15 items
- Intrapersonal Strength -- outlook on competence and accomplishments -- 11 items
- Affective Strength -- ability to accept affection from and express feelings toward others -- 7 items
- Family Involvement -- participation in and the relationship with family -- 10 items
- School Functioning -- competence in school and classroom tasks -- 9 items
Career Strength Scale -- This measured interest and aptitude for career and vocational development -- 5 items
Youth Mentor Relationship Questionnaire -- 20 item scale with four subscales "not dissatisfied", "helped to cope", "not unhappy" "trust not broken". This was measured on a scale of 1-4 with higher scores corresponding to more successful relationships.
Several variables identify delinquent behavior. The Utah Administrative Office of the Courts provided information on each incident which came to the attention of the court for each youth, including the incident date, charges, intake decision, and conviction. Self-reported delinquent behavior was collected in the pre- and post-tests on the number of times the behavior occurred.
Socioeconomic and family status questions ranging from whether the youth qualifies for free lunch, who is living with the youth, government services that are being received are also present in the data. Responses are reported through scales and direct wording used by the youth.
Date variables -- Date variables were collected for pre- and post-test completion, discharge date, and study discharge date.
School Data -- School absences and GPA records.
Court Data recording delinquent incidences.
Analysis comparing pre- and post-test score changes for BERS-2 scale.
Variables considering the matching of youth to mentor, including the reason the match occurred, the date the match occurred, and whether the youth unmatched to the mentor and why.
Variables concerning the date the youth and mentor manuals were issued, and the date the parent was given their guide.
Variables considering the type of mentor that the youth was given, and whether the mentor attended orientation.
Data Set 2 consists of 91 cases and 104 variables representing survey data from the YFP program mentors. Questions focused on the mentor's length of service in YFP, the matching that took place with the mentee, interactions with their mentee, and their evaluation of mentee success.
Response Rates View help for Response Rates
The response rate varied by data collection point. There were 401 youths in the treatment group and 327 in the comparison group.
Experimental group
- Pre-test: 392
- Post-Test 1 Only (no Pre-Test): 9
- Pre-Test and Post-Test 1: 270
- Pre-Test and Post-Test 2: 98
- Pre-Test, Post-Test 1 and Post-Test 2: 93
- Post-Test 3: 28
- Follow-up interview: year one: 40 percent, year 2: 31 percent, year 3: 30 percent
- School data: 39 percent
- Court data: 82 percent
Comparison Group
- Pre-test: 327
- Pre-Test and Post-Test 1: 280
- Pre-Test and Post-Test 2: 144
- Pre-Test, Post-Test 1 and Post-Test 2: 131
- Follow-up interview: year one: 37 percent, year 2: 49 percent, year 3: 36 percent
- School data: 73 percent
- Court data: 98 percent
Presence of Common Scales View help for Presence of Common Scales
Behavioral and Emotional Rating Scale (BERS) - 2nd edition Youth-Mentor Relationship Questionnaire (YMRQ)
HideOriginal Release Date View help for Original Release Date
2015-12-23
Version History View help for Version History
- Orchowsky, Stan, and Mary Poulin. A Process and Outcome Evaluation of the 4-H Mentoring/Youth and Families with Promise (YFP) Program [Utah, 2005-2010]. ICPSR33781-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2015-12-23. http://doi.org/10.3886/ICPSR33781.v1
2016-01-13 Updated User Guide.
2015-12-23 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:
- Created variable labels and/or value labels.
- Standardized missing values.
- Performed recodes and/or calculated derived variables.
- Checked for undocumented or out-of-range codes.
Notes
The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.
One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.
This dataset is maintained and distributed by the National Archive of Criminal Justice Data (NACJD), the criminal justice archive within ICPSR. NACJD is primarily sponsored by three agencies within the U.S. Department of Justice: the Bureau of Justice Statistics, the National Institute of Justice, and the Office of Juvenile Justice and Delinquency Prevention.