Behavioral Interventions to Advance Self-Sufficiency (BIAS) Project, United States, 2014-2015 (ICPSR 38319)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Data Collection Notes View help for Data Collection Notes

Hide

Study Purpose View help for Study Purpose

Ohio

The BIAS project in Ohio sought to increase child support payments from non-custodial parents (NCPs) who were not having their wages garnished.

Indiana

The Behavioral Interventions to Address Self-Sufficiency (BIAS) project in Indiana focused on child care decision making among low-income parents and sought to improve on-time child-care subsidy renewal rates. The project aimed to increase the percentage of parents who used their Child Care and Development Fund (CCDF) subsidies to pay for highly rated providers in the state's quality rating and improvement system - Paths to QUALITY (PTQ). The PTQ program ranks child care providers on a four-point scale based on standards related to health and safety, staff qualifications, parental engagement, and curriculum development. Additionally, parents have to verify their eligibility for the CCDF at least every six months. On-time renewals ensures continuation of benefits and thus consistent child-care for families, and stable payments for providers. The BIAS intervention diagnosed factors that might inhibit smooth renewal and designed an intervention for improvement: a 'client intervention', which used early and clear communications to encourage parents to attend their first appointment, and clarified the renewal process and documents needed to do so.

Los Angeles

The BIAS project in Los Angeles sought to improve the number of Temporary Assistance for Needy Families (TANF) recipients who 'reengaged' in the county's welfare-to-work programs. Some adults had received an exemption that ended in 2013 that ended due to state policy changes; the county made attempts to bring these parents into the welfare-to-work program and engage participants. The county engaged in a mailing campaign to boost re-engagement; in spite of this, only about half of participants attended the scheduled re-engagement appointment. The BIAS intervention sent additional behaviorally informed materials to participants one week before the appointment: a 'gains framed notice', which highlighted the benefits to parents to encourage parents to attend their first appointment, or a 'loss framed notice' which highlighted what parents stood to lose if they did not attend their appointment.

Oklahoma

The BIAS project in Oklahoma sought to improve on-time child-care subsidy renewal rates. On-time renewals ensures continuation of benefits and thus consistent child-care for families, and stable payments for providers.

Paycheck Plus (New York City)

The BIAS project in New York sought to improve the number of participants who attended an informational meeting about a program they had enrolled in called the Paycheck Plus Demonstration, which was a program modelled on the Earned Income Tax Credit, and was offered to single New Yorkers. The city engaged in a marketing campaign to boost participation.

Study Design View help for Study Design

Ohio

In order to test whether these changes were effective in increasing the obligor's child support payments, obligors were randomly assigned to behaviorally-informed treatments or a business-as-usual control group. Experimental conditions differed in each sample.

The BIAS team mapped the steps an obligor needs to follow and the decisions an obligor needs to make in order to make a child support payment, focusing on the bottlenecks that emerged at each step. Interventions were designed to mitigate these potential "hassles." The original BIAS Child Support experiment ran in Franklin County and included a behaviorally-informed notice, reminder phone calls, and early and late payment due dates. A follow-up study was conducted in Cuyahoga County based on lessons learned in Franklin County. The Cuyahoga study was able to detect smaller effects due to a larger sample size and longer follow up period, and included a more-refined notice and text message reminders.

Indiana

For the part of the Indiana project that tracked Child Care and Development Fund (CCDF) subsidies, intervention materials were mailed to families from June to October 2014; the BIAS team tracked parents' child care choices for those who enrolled in the CCDF program for eight additional months. There were three study groups: a control group that received the standard letter and brochure, an intervention group that received personalized referrals by mail (referrals-only group), and another intervention group that received personalized referrals and a phone call from staff (referrals-phone group). The evaluation estimated the effects of the referrals-only and referrals-phone groups, compared with the control group, on the following primary outcomes:

• Selection of a Paths to QUALITY (PTQ) provider at the time of enrollment
• Selection of a highly rated PTQ provider (Level 3 or 4) at the time of enrollment

For the part of the Indiana project that focused on renewal rates, clients were randomly assigned to the program, and the remainder served as a 'business-as-usual' control group - a counterfactual, which did not receive the modified BIAS mailing, and thus provided a benchmark indicating what would have happened in the absence of the program. The evaluation also included an assessment of the program's implementation. Two rounds of redetermination tests were conducted in Indiana, an example of rapid-cycle evaluation. The first test was launched in January 2014 and the second in June 2014, based on findings from Round 1.

Los Angeles

In order to test conclusively whether or not BIAS increased reengagement rates, more than 2,000 clients were randomly assigned to one of three groups. Clients could be assigned to the program group that the received the 'gain framed notice', or the program group that received a 'loss framed notice'; the remainder served as a 'business-as-usual' control group - a counterfactual, which did not receive the modified BIAS intervention, and thus provided a benchmark indicating what would have happened in the absence of the program. By randomly dividing the sample into these three groups, the study was able to test conclusively whether or not the re-designed client materials encouraged its participants to attend their appointment. The evaluation also included an assessment of the program's implementation.

Oklahoma

The BIAS intervention diagnosed factors that might inhibit on-time renewal and designed three interventions for improvement: (1) a 'provider intervention' which gave child care providers more information about their clients renewal deadlines; (2) a 'client intervention', which used early and clear communications to clarify the renewal process and continued follow-up communications; and (3) a 'combined intervention', which included both the client and provider interventions.

There were two groups which were randomly assigned - provider and clients - and the evaluation was done using a factorial design. In order to test conclusively whether or not BIAS increased renewal rates, more than 9,000 clients were randomly assigned to the program, and the remainder served as a 'business-as-usual' control group - a counterfactual, which did not receive the modified BIAS mailing, and thus provided a benchmark indicating what would have happened in the absence of the program. Providers for these randomly assigned clients were then randomly assigned to provider program and control groups. As a result of this, there were four combinations of providers and clients: those who were only receiving the client intervention, those who were only receiving the provider intervention, those who were receiving both, and those receiving none (the control). By randomly dividing the sample into these groups, the study was able to test conclusively whether or not the re-designed client or provider materials encouraged its participants to renew more frequently and in a timelier manner. The evaluation also included an assessment of the program's implementation, and a cost-benefit analysis.

Paycheck Plus (New York City)

In order to test conclusively whether or not BIAS increased reengagement rates, more than 3,000 clients were randomly assigned to one of two rounds, and then to evaluation groups. In round 1, clients could be assigned to the program group that received the two postcards and weekly text messages, or the program group that received only the two postcards; the remainder served as 'business-as-usual' control groups - one that received standard postcards, and another that received two postcards and text messages with "standard" messaging. These groups served a counterfactual, which did not receive the modified BIAS intervention, and thus provided a benchmark indicating what would have happened in the absence of the program. By randomly dividing the sample into these four groups, the study was able to test conclusively whether or not the re-designed client materials encouraged its participants to attend their appointment. In round 2, any clients who did not participate in a meeting in round 1 were randomly assigned to one of two groups. Clients could be assigned to the program group, which were invited to attend their meeting on Paycheck Plus over the phone, or they were assigned to the control group which attended their meetings on Paycheck Plus in person. The control group again served as a counterfactual and was used to test conclusively whether or not the meeting type encouraged participants to attend appointments.

Sample View help for Sample

Ohio

The Franklin County experiment consisted of two samples. Sample 1 included obligors who had a current order, were not currently paying through wage withholding by an employer, and did not receive any payment notice from the state at the time of random assignment. 15,715 obligors were in this group. Sample 2 included obligors who had a current order, were not currently paying through wage withholding by an employer, but did receive any payment notice from the state. 10,741 obligors were in this group.

The Cuyahoga County test demonstrated that obligors who received a reminder (either notice or phone call) were more likely to make a payment during follow-up than obligors who did not receive any reminder. However, it showed no significant difference between the type of reminder (payment reminder notice or robo-call), and no better effect with a bundled intervention of the two types of reminders. Within the month, the timing of when the notices were sent or when the due date was set also did not show a differential impact. For these reasons, we did not replicate these specific elements of the Franklin County test.

For more information about each sample within the Franklin and Cuyahoga County tests, please consult the P.I. Documentation for Ohio Child Support Data.

Indiana

For Indiana Child Care Subsidy Round 1 Data, clients were randomized to either a program or control group. In total, the random assignment produced a sample of 5,332 members. Clients with renewals from January-June 2014 were randomly assigned for Round 1.

For Indiana Child Care Subsidy Round 2 Data, clients were randomized to either a program or control group. About 20 percent of the families in the sample went through the redetermination process more than once during the study period and were assigned to their original group for subsequent assignments. The report analysis only reports on the family's initial recertification process. Clients with renewals in July-November 2014 were randomly assigned for Round 2. The random assignment produced a total sample of 5,686 members and 4,732 unique members who went through at least one cycle.

For Indiana Quality Rated Child Care Data, there were three study groups: a control group that received the standard letter and brochure, an intervention group that received personalized referrals by mail (referrals-only group)and another intervention group that received personalized referrals and a phone call (referrals-phone group). In total, the random assignment produced a sample of 12,652 children. Because the BIAS team randomly assigned Cohort 1 in one week, it limited the number of parents assigned to the referrals-phone group to ensure IACCRR staff could conduct all the personal phone calls within the intended timeframe. As a result, there are 4,255 children in the control group, 1,191 in the referrals-only group, and 503 in the referrals phone group. Cohort 2 includes about an equal number of children in each study group: 2,402 in the control group, 2,389 in the referrals-only group, and 1,912 in the referrals-phone group.

For more information about each sample within the Indiana tests, please consult the P.I. Documentation for Indiana Child Care Subsidy Rounds 1 and 2 Data and/or the P.I. Documentation for Indiana Quality Rated Child Care Data.

Los Angeles

Clients were randomized to either a program or control group. In total, the RA produced a sample of 2,442 members. Clients who were mandated to attend a reengagement appointment between July 2014 and November 2014 were randomly assigned.

For more information about sampling within the Los Angeles tests, please consult the P.I. Documentation for Los Angeles Data.

Oklahoma

There were two randomizations performed; one for the clients who were receiving childcare subsidies, and one for the providers who were working with clients with subsidies. In total, the client-RA produced a sample of 9,348 members. Clients with renewals in June, July, or August were randomly assigned; in order to protect client confidentiality, their date of renewal is not present in the data. All date and time information is provided in a relative context. Providers were randomized based upon current and expected client enrolment and later constrained based upon actual enrolment. The final provider-RA sample was 1,035 providers.

For more information about sampling within the Oklahoma tests, please consult the P.I. Documentation for Oklahoma Data.

Paycheck Plus (New York City)

Clients were randomized to either a program or control group from September 27th 2013 to February 18th 2014. In total, the RA produced a sample of 3,014 program members; all 3,014 members were RAed in Round 1, and then 2,351 members were subsequently RAed in Round 2. There were a number of clients who withdrew from the study and were thus not included in the impact sample; the variables used to denote this are documented above. Clients who were asked to attend a Paycheck Plus meeting were randomly assigned.

For more information about sampling within the Paycheck Plus tests, please consult the P.I. Documentation for Paycheck Plus Data.

Time Method View help for Time Method

Unit(s) of Observation View help for Unit(s) of Observation

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Hide

Original Release Date View help for Original Release Date

2022-05-19

Hide