Evaluation of Employment Coaching for Temporary Assistance for Needy Families (TANF) and Related Populations & Long-Term Follow-Up Study, 7 U.S. states, 2016-2026 (ICPSR 39080)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Data Collection Notes View help for Data Collection Notes

Study Purpose View help for Study Purpose

Policymakers, practitioners, researchers, and others are interested in the potential of employment coaching to help recipients of Temporary Assistance for Needy Families (TANF) and other adults with low incomes to become economically secure. Employment coaching involves trained staff working collaboratively with TANF participants to help them set individualized goals that are directly or indirectly related to employment. Coaches provide motivation, support, and feedback as participants work toward those goals. Unlike most traditional case managers, coaches work in partnership with participants. They do not tell participants what goals to set or what actions to take to work toward them. Rather, participants are given autonomy to identify and pursue their goals, with support from their coach. This nondirective approach to goal setting can help people use and strengthen the skills needed to stay organized, finish tasks, and control emotions, which are referred to as self-regulation skills. These skills are important for obtaining, keeping, and advancing in a job. Yet, poverty and other chronic stressors can hinder the use of these skills. By helping participants practice these skills, coaching could improve employment outcomes, and hence economic security.

To explore the potential of employment coaching for adults with low incomes, the Office of Planning, Research, and Evaluation in the ACF, U.S. Department of Health and Human Services, contracted with Mathematica, Abt Associates, MDRC, and The Adjacent Possible to conduct the Evaluation of Employment Coaching for TANF and Related Populations. This evaluation builds the evidence base by rigorously testing four employment coaching programs designed for adults with low incomes. It assesses the implementation of the four coaching programs and--via an experimental study--their impacts on study participants' self-regulation skills, employment, earnings, self-sufficiency, and other measures of personal and family well-being.

Study Design View help for Study Design

Researchers used an experimental design to assess the effectiveness of each employment coaching program in improving participants' outcomes. Between February 2017 and November 2019, about 4,300 adults who were eligible for one of the four employment coaching programs included in the evaluation and who consented to participate in the study were randomly assigned with equal probability either to a program group that was given access to employment coaching or to a control group that did not have access to employment coaching from the program. In the impact study of Goal4 It!, control group members were provided TANF case management. In the study of FaDSS, all program and control group members received TANF case management. In the study of all four programs, all study participants, whether in the program or control group, could receive other services available in the community.

To estimate the impacts of employment coaching, the study collected outcomes from survey and administrative data. The effectiveness of each employment coaching program was assessed based on differences in average outcomes between members of the program and control groups. With random assignment, the program and control group members had similar characteristics and experiences, on average, before participating in the program so any differences in observed outcomes could be attributed to employment coaching. Researchers estimated the impact on each outcome using a statistical model to control for baseline characteristics and improve the precision of the impact estimates. For each impact estimate, researchers reported whether the difference from zero was statistically significant. For impacts on earnings, researchers complemented this by using a Bayesian analysis approach to estimate the probability that the program's impact was than a specified amount.

Sample View help for Sample

The study teams at each employment coaching program location determined participant eligibility and obtained consent before enrolling eligible participants in the study and then conducting random assignment. Study enrollment and random assignment across all programs took place from February 2017 to November 2019, although different programs conducted enrollment at different times. Across the four programs, 4,276 adults who were eligible for one of the four employment coaching programs and who consented to participate in the study were randomly assigned either to a program group that had access to the coaching program or a control group that did not have access to the coaching program.

1. FaDSS: To be eligible to enroll in FaDSS, participants must be receiving cash assistance from the Family Investment Program, Iowa's TANF program. FaDSS aims to serve TANF recipients who are determined by their case manager to be at risk of long-term dependency on TANF. Most study participants (80 percent in 2019, the last year of study enrollment) were referred to FaDSS by their TANF case manager. Almost all of the remaining 20 percent contacted the program directly after learning about it from a local service provider, relatives, or friends. From June 2018 to November 2019, 863 adults enrolled in the study. All study applicants who were found eligible for the program and consented to participate in the study were randomly assigned to either the FaDSS group and offered FaDSS services, or to a control group and not offered FaDSS services. Both the FaDSS and control groups were required to receive case management as part of Iowa's TANF Employment and Training program, and both groups could access other services in the community.
2. Goal4 It!: To be eligible to receive Goal4 It!, participants must be deemed eligible for TANF cash assistance in Jefferson County, Colorado, and be subject to TANF work requirements. Participants can receive Goal4 It! only while they are participating in TANF. From October 2018 to November 2019, 802 adults enrolled in the study. All TANF recipients who were subject to the TANF work requirements and consented to participate in the study were randomly assigned to either the Goal4 It! group or the control group. Members of the Goal4 It! group were required to receive coaching using Goal4 It!. Control group members could not access Goal4 It! coaching but were required to participate in traditional case management from a TANF case manager. Similar to Goal4 It!, traditional case management incorporated setting goals. But unlike Goal4 It!, in which participants determined goals, case managers from the traditional case management approach typically directed goal setting and identified steps for participants. Members of both the Goal4 It! and control groups could access other services in the community.
3. LIFT: To enroll in LIFT, applicants must be parents or other caregivers of children younger than age 8, or expectant parents. They also must demonstrate the level of stability needed to work on long- and short-term goals, meaning they must have had stable housing for at least six months and (1) be employed or live with someone who is employed at least part time, or (2) be in an educational program. From June 2018 to November 2019, 808 adults enrolled in the study. All applicants who were found eligible for the study and consented to participate in the study were randomly assigned to either the LIFT group, who could participate in LIFT, or the control group, who could not participate in LIFT. Members of both the LIFT and control groups could access other services available in the community.
4. MyGoals: To be eligible to enroll in MyGoals, participants must have been an adult member of a household receiving federal housing assistance (through the Housing Choice Voucher program or living in public housing) and either have been unemployed or working fewer than 20 hours per month. Participants also must have been legally able to work in the United States and must not have been participating in Jobs Plus or the Family Self-Sufficiency Program, which were also managed by the public housing agencies and provided similar services. Study participants were referred to MyGoals by the housing agencies in Baltimore and Houston, or they learned about the program from a MyGoals coach at a recruitment event. MyGoals coaches conducted outreach at community locations such as job fairs, libraries, and TANF offices, and at public housing developments in Baltimore. From February 2017 to November 2019, 1,803 adults enrolled in the study. All study applicants who were found eligible for the study and consented to participate in the study were randomly assigned to either the MyGoals group, who could participate in MyGoals, or a control group, who could not participate in MyGoals. Both the MyGoals group and control group members could access other services available in the community.

Time Method View help for Time Method

Universe View help for Universe

TANF recipients and other low-income individuals living in: Iowa; Jefferson County, Colorado; Los Angeles, California; New York City, New York; Baltimore, Maryland; and Houston, Texas.

Unit(s) of Observation View help for Unit(s) of Observation

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Description of Variables View help for Description of Variables

The study team collected baseline data on participants' demographics, employment, education level, family background, household composition, earnings, receipt of public assistance, and goal-setting and self-regulation skills. The study team then contacted participants to complete two follow-up surveys that collected information on an extensive set of outcomes, including service receipt, labor market outcomes such as employment and earnings, economic security, receipt of public assistance, and self-regulation skills. TANF and SNAP administrative records containing information on baseline and follow-up receipt of public assistance benefits were also collected from state agencies. As previously noted, the study team also analyzed administrative earnings and employment data from the NDNH that are not part of the restricted files.

Although the study examines each program's impact on a broad set of outcomes, it focuses on a few key outcomes that the program is expected to change, referred to as confirmatory outcomes. The main tests of the programs' effectiveness are based on whether the program had a favorable impact on the confirmatory outcomes. The confirmatory outcomes include measures in three domains for all four programs for both follow-up analysis and a fourth domain for FaDSS and Goal4 It! for the second follow-up analysis:

1. Self-regulation and goal-related skills. Setting goals and working to attain them requires self-regulation skills and is the centerpiece of employment coaching. Researchers used an eight-item scale on goal-setting and attainment skills designed to measure people's ability to set and work toward attaining employment goals as a confirmatory outcome.
2. Labor market outcomes. Researchers used earnings as the confirmatory measure of labor market success because they encompass three ways that employment coaching could influence labor market success: obtaining a job, working more regularly or more hours, or earning higher wages. Researchers measured earnings using both responses to the follow-up survey and NDNH administrative records. Earnings reported on the survey cover all jobs the study participant may have had, but may be subject to error if study participants remember jobs incorrectly.
3. Economic well-being. All programs in the study intend to improve economic well-being. This may be accomplished through improved labor market outcomes, access to other material supports (such as assistance programs), or better financial management. Researchers used a six-item economic hardship scale to assess the extent to which scarce economic resources affected key aspects of material well-being, such as food, housing, and medical care.
4. Receipt of public assistance. The FaDSS and Goal4 It! programs both include TANF benefit receipt among their enrollment criteria. Reducing participation in TANF is a goal of these programs. Because impacts on TANF benefit receipt should emerge after impacts on earnings, TANF benefit receipt for the second follow-up period is included as a confirmatory outcome for these two programs. Researchers measured receipt of public assistance using public assistance agency administrative records on average monthly TANF benefits.

The study team also examined the impact of each program on other outcomes not deemed as confirmatory. Examples of these outcomes include the receipt of other employment services; participation in, completion of, and receipt of credentials from training and education programs; employment.

Key study variables used in the impact analysis were constructed from one or more raw study variables. Constructed variables include scales that take the average value of responses to multiple survey items, indicator variables using simple combinations of multiple survey items, combinations of variables that provided the same type of information, and earnings and employment variables constructed from participants' reported employment timelines.

Response Rates View help for Response Rates

FaDSS

• First follow-up survey response rate: 63 percent for both program group and control group.
• Second follow-up survey response rate: 76 percent for both program group and control group.
• Administrative data on TANF and SNAP benefits match rate: 99 percent for both program group and control group.

Goal4 It!

• First follow-up survey response rate: 59 percent (program group) and 58 percent (control group).
• Second follow-up survey response rate: 71 percent for both program group and control group.
• Administrative data on TANF and SNAP benefits match rate: 100 percent for both program group and control group.

LIFT

• First follow-up survey response rate: 77 percent (program group) and 73 percent (control group).
• Second follow-up survey response rate: 87 percent (program group) and 83 percent (control group).
• Administrative data on TANF and SNAP benefits match rate: 75 percent (program group) and 74 percent (control group).

MyGoals

• First follow-up survey response rate: 77 percent (program group) and 73 percent (control group).
• Second follow-up survey response rate: 82 percent (program group) and 79 percent (control group).
• Administrative data on TANF and SNAP benefits match rate: 100 percent for both program group and control group.

Presence of Common Scales View help for Presence of Common Scales

The study team constructed scale variables by taking the average value of responses to multiple survey items. The study team coded respondents as missing scale variables if they responded to fewer than two-thirds of the items used to construct a scale.

Constructed scale variables have variable labels that indicate they are scales (for instance, "Goal-setting and attainment scale").

Original Release Date View help for Original Release Date

2025-01-13

Version History View help for Version History

Weight View help for Weight

To account for participants who (1) did not respond to the first follow-up survey, (2) did not respond to the second follow-up survey, and/or (3) did not match to the administrative data, the study team constructed three separate nonresponse weights for the impact analyses. These weights were used to adjust the data by multiplying each respondent's data by their weight. This has the effect of giving more weight to the responses of groups who were less likely to respond to a survey or match to administrative records, and less weight to the responses of groups that had a higher response rate or were more likely to match to administrative records, so that the impact analysis sample is more representative of the full randomly assigned sample.

The study team constructed each weight by estimating a series of logistic regression models that predicted (1) survey nonresponse or (2) inability to match to the administrative data for each employment coaching program and research group. Each participant was then assigned a weight corresponding to the probability that they (1) responded to a given follow-up survey (and/or (2) were matched to administrative data on public benefit receipt. Participants who did not respond and/or were not matched to the administrative data were assigned a weight equal to zero.