Multi-Site Evaluation of Reduced Probation Caseload Size in an Evidence-Based Practice Setting in Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado, 1997-2010 (ICPSR 31961)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Data Collection Notes View help for Data Collection Notes

Study Purpose View help for Study Purpose

The purpose of this study was to address whether reduced caseloads improve probation outcomes in agencies that have implemented evidence-based practices (EBP). Another purpose was to examine how probation officers in the study have implemented supervision strategies associated with EBP and if changes in outcomes could be observed over time as components of EBP are introduced.

Study Design View help for Study Design

The Oklahoma City, Oklahoma evaluation (Datasets 1 and 2) began with a randomized controlled trial (RCT) where probation officers volunteered to take part in the study. The RCT was established after determining that Oklahoma City had sufficiently implemented evidence-based practices (EBP) to provide the necessary context for the study. Half of the probation officers were assigned to an experimental condition with a reduced caseload. The rest were assigned to a control condition and the probation officers that did not volunteer comprised a comparison group. Over time the experiment degenerated and the study combined the control and comparison groups into a single comparison group and applied a difference-in-difference estimator to study treatment effects. Data were collected from probation records and criminal histories. Pre-implementation data were collected on 43,688 probation cases and post-implementation data were collected on 5,905 cases.

A regression discontinuity design was used in Polk County, Iowa (Dataset 3). A multi-year cohort of data from Polk County was used to estimate the effects of reduced caseloads on criminal recidivism. The data, provided by the Iowa Department of Corrections, include probation and court information for 3,254 probationers under supervision during the years 2001-2007.

The study team initially sought to implement RCT in several judicial districts in Colorado (Dataset 4). However, the resources of some jurisdictions did not allow for implementation of a design that required maintenance of reduced caseloads for an extended period of time. Thus, the four largest districts in Colorado with a total of 111,523 probation cases participated in the regression discontinuity design study (RDD). The state Division of Probation Services provided the study team with a multi-year cohort of probation data, from 1997-2007. Outcome data were derived from court filings data provided by the Colorado State Court Administrator's Office.

All probation officers in the study completed audio tapes of their supervision sessions and completed a battery of four survey instruments at two points in the study, with a year interval in between (Dataset 5). The tapes were analyzed by independent raters trained in the three different recognized systems for behavioral rating of officer skills: (1) Motivational Interviewing Therapeutic Integrity-2 (MITI-2); (2) The Dual Role Inventory-Revised (DRI-R); and (3) The Strategic Initiative for Community Supervision (STICS). These rating systems focus respectively on Motivational Interviewing (MI) skills, balance in required roles and the content - criminogenic needs versus non-criminogenic needs - focus of the supervision session. The survey data were derived from four survey tools (Likert Organizational Climate Scale; Intrinsic Motivation Inventory; Probation and Parole Strategies Questionnaire; and, the Employee Satisfaction Survey) previously used in probation survey studies. Measures of relevant constructs were established using data from 72 probation officers including 18 officers in Iowa, 22 officers in Oklahoma, and 32 officers in Colorado at Wave 1. At Wave 2, data were available from the same 18 officers in Iowa, 15 of the original 22 officers in Oklahoma, and 29 of the 32 officers in Colorado.

Sample View help for Sample

The study team screened over two dozen sites to determine their eligibility and appropriateness for the study. Sites were first screened via interviews and agency documentation for general eligibility. Critical criteria for eligibility in the study were a sufficient sample size of probationers and probation officers; adequate progress made in implementation of evidence-based practices (EBP); minimum data availability; and general willingness to participate. Sites that met these initial criteria were invited to participate in a more rigorous screening process. There were three sites that met this criteria: Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado.

The Oklahoma City Evidence-Based Practices Pre-Implementation Data (Dataset 1) are comprised of 43,688 probation cases and the Oklahoma City Evidence-Based Practices Post-Implementation Data (Dataset 2) are comprised of 5,905 probation cases. The Polk County, Iowa Data (Dataset 3) are comprised of an overall sample size of 3,254 cases. A total of 111,523 cases from the four largest districts in Colorado are included in the study (Dataset 4). In the Evidence-Based Practices Probation Officer Data (Dataset 5), a convenient sample of 72 probation officers was drawn from three state probation systems using a sampling frame that split the total sample between officers with small caseloads and officers with regular caseload sizes.

Time Method View help for Time Method

Universe View help for Universe

Datasets 1 and 2: All probation cases in Oklahoma City, Oklahoma between 2001 and 2010.

Dataset 3: All probation cases in Polk County, Iowa between 2001 and 2007.

Dataset 4: All probation cases in Colorado between 1997 and 2007

Dataset 5: All probation officers in Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado between 2007 and 2010.

Unit(s) of Observation View help for Unit(s) of Observation

Data Source View help for Data Source

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Description of Variables View help for Description of Variables

The Oklahoma City Evidence-Based Practices Pre-Implementation Data (Dataset 1) contain a total of 14 variables including probation supervision information and demographic characteristics. Probation supervision information includes Level of Service Inventory (LSI) Scores, time on supervision, reasons for termination of probation, and failure of probation. Demographic characteristics include employment, sex, education, prior convictions, race, and whether or not probationer had any substance use or abuse.

The Oklahoma City Evidence-Based Practices Post-Implementation Data (Dataset 2) contain a total of 110 variables including demographic characteristics, history of alcohol or drug abuse, criminal records, and probation information. Demographic characteristics include sex, employment, education, age, and race. History of alcohol or drug abuse measured whether or not probationer abused drugs and alcohol and the severity of abuse. Criminal record information includes number of prior arrests, incarcerations, and probations. Probation information includes whether or not probationer had any probation violations or new convictions, date of arrest offense, number and rate of contacts probationer had with probation officers and probation related services, level and status of mental health, alcohol and drug treatment for probationer.

The Polk County, Iowa Data (Dataset 3) contain a total of 327 variables including demographic characteristics, criminal records, and probation information. Demographic characteristics include sex, race, ethnicity, marital status, education, employment, age, and number of dependents. Criminal record information includes prior juvenile convictions, prior probation revocation, type of prior convictions and year of conviction, number and type of prior offenses. Probation information includes start and end dates of supervision, age at start of supervision, date to first arrest, whether there was a revocation of probation and reason for revocation, reason for change in supervision status, re-arrest charge description and type, number and place of contacts probationer had with probation officers, the Iowa Risk score, which determines the supervision intensity, new arrest charge, whether treatment was needed and completed, type of treatment, risk assessment dates, intervention dates, descriptions, categories, settings, length, and reason for termination.

The Colorado Data (Dataset 4) contain 119 variables including demographic characteristics, probation information, treatment information, and arrest data. Demographic characteristics include age at beginning of supervision, race, sex, date supervision started, number of days under supervision, and whether termination codes were found in probation term. Probation information includes number of days first revocation code found during supervision term, revocation codes found in probation term, number of assessments found in probation term, number of probation officer records found for probation term, supervision level found for probation term, number and rate of collateral contacts, general casework, face to face contacts, no show appointments, victim contacts, office visits, employment verification, domestic violence screenings, curfew checks, home visits, and parent or guardian contact within supervision term. Treatment information includes whether urinalysis treatment records were found, number of urinalysis treatment records found, any treatment records with description found, and number of treatment records found. Arrest data includes number of days between arrest and beginning of supervision, arrest offense, and categorization of arrest offense.

The Evidence-Based Practices Probation Officer Data (Dataset 5) contain a total of 61 variables including data collection site, years of education, and a number of probation officer attitudinal and casework skill variables.

Response Rates View help for Response Rates

Not applicable

Presence of Common Scales View help for Presence of Common Scales

The Evidence-Based Practices Probation Officer Data (Dataset 5) contain survey measures, evidence-based practice (EBP) skill measures, measures of officer supervision session content, and process/outcome measures.

Survey measures include the Intrinsic Motivation Inventory (IMI), the Enforcing Compliance subscale of the IMI, the Probation and Parole Strategies Questionnaire survey (PPSQ), an organizational climate measure, a positive attitude score derived by combining each officer's total scores on the Likert Organizational Climate Scale (LOCS) and the Employment Satisfaction Survey (ESS), as well as law enforcement, case worker, and resource broker subscale orientation measures of the PPSQ.

EBP skill measures include relative Skill Balance (SKB), a global rating composite, a Motivational Interviewing Therapeutic Integrity (MITI) average composite behavioral rating, an Officer Interactive Skills (OIS) measure, a Quality Contact Standards (QCS) score, an officer demeanor score from the Department and Manner of Being subscale of the QCS, an assessment and planning skills score from the Assessment and Planning subscale of the QCS, an engagement in treatment referral score from the Referral for Treatment and Services subscale of the QCS, an enforcement of sanctions score from the Enforcement of Sanctions and Ground Rules QCS subscale, an overuse of questions measure, the Dual Role Inventory-Revised score, and a standardized summary measure of officer skills.

Measures of officer supervision session content that emerged in the training and study for the Canadian Strategic Training Initiative in Community Supervision (STICS) include percentage of supervision session time spent discussing a criminogenic need and percentage of time an officer spends discussing non-criminogenic needs in their taped supervision session. Additional measures of officer supervision session content include the percentage of time in supervision session that an officer spends discussing terms and conditions of supervision, duration of each officer's taped supervision session (measured in minutes), a composite skill and supervision measure, and a hybrid skill measure and supervision content score.

Process/outcome measures include Officer Change Talk (a performance score to assess each officer's influence on the offender), Preparatory Change Talk, and Commitment Change Talk.

Original Release Date View help for Original Release Date

2014-02-14

Version History View help for Version History

Weight View help for Weight

None