Multi-Site Evaluation of Reduced Probation Caseload Size in an Evidence-Based Practice Setting in Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado, 1997-2010 (ICPSR 31961)

Published: Feb 14, 2014

Principal Investigator(s):
William Rhodes, Abt Associates

Version V1

Criminal justice researchers have studied caseload size to determine whether smaller caseloads improve probation outcomes. The purpose of this study was to investigate whether reduced caseloads improved supervision outcomes for medium to high risk offenders in a probation agency that trains its officers to apply a balance of controlling and correctional/rehabilitative measures. Three different probation agencies were selected to take part in this study. The first was Oklahoma City (Datasets 1 and 2), where a randomized controlled trial (RCT) experiment was implemented. The second site was Polk County, Iowa (Dataset 3), where a regression discontinuity design study (RDD) was implemented. Lastly, four judicial districts in Colorado (Dataset 4) were selected where a RDD study again was implemented. In Oklahoma City the RCT degenerated and the study team turned to a difference in differences (DD) estimator. The research team also collected Evidence-Based Practices Probation Officer Data (Dataset 5) from a set of survey questionnaires administered to the participating officers from the three research sites and the scoring of taped officers' supervision sessions with probationers on their caseloads by trained raters.

Rhodes, William. Multi-Site Evaluation of Reduced Probation Caseload Size in an Evidence-Based Practice Setting in Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado, 1997-2010. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2014-02-14.

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

United States Department of Justice. Office of Justice Programs. National Institute of Justice (2006-IJ-CX-0011)

Datasets 1, 2, and 3: None; Dataset 4: District; Dataset 5: site (state)

Access to these data is restricted. Users interested in obtaining these data must complete a Restricted Data Use Agreement, specify the reasons for the request, and obtain IRB approval or notice of exemption for their research.

2001 -- 2010 (Oklahoma City, Oklahoma Data)

2001 -- 2007 (Polk County, Iowa Data)

1997 -- 2007 (Colorado Data)

2007 -- 2010 (Evidence Based Practices Probation Officer Data)

2007 -- 2010

Qualitative data from probation agencies and officers discussed in the final report (Kuck Jalbert et al., 2011; NCJ 234596) are not available as part of this data collection.

Additional contributions to this study were made by the following individuals: Sarah Kuck Jalbert (Abt Associates, Inc.), Michael Kane (Criminal Justice Institute), Elyse Clawson (Criminal Justice Institute), Bradford Bogue (Justice System Assessment and Training), Chris Flygare (Abt Associates, Inc.), Ryan Kling (Abt Associates, Inc.), and Meaghan Guevara (Criminal Justice Institute).

The purpose of this study was to address whether reduced caseloads improve probation outcomes in agencies that have implemented evidence-based practices (EBP). Another purpose was to examine how probation officers in the study have implemented supervision strategies associated with EBP and if changes in outcomes could be observed over time as components of EBP are introduced.

The Oklahoma City, Oklahoma evaluation (Datasets 1 and 2) began with a randomized controlled trial (RCT) where probation officers volunteered to take part in the study. The RCT was established after determining that Oklahoma City had sufficiently implemented evidence-based practices (EBP) to provide the necessary context for the study. Half of the probation officers were assigned to an experimental condition with a reduced caseload. The rest were assigned to a control condition and the probation officers that did not volunteer comprised a comparison group. Over time the experiment degenerated and the study combined the control and comparison groups into a single comparison group and applied a difference-in-difference estimator to study treatment effects. Data were collected from probation records and criminal histories. Pre-implementation data were collected on 43,688 probation cases and post-implementation data were collected on 5,905 cases.

A regression discontinuity design was used in Polk County, Iowa (Dataset 3). A multi-year cohort of data from Polk County was used to estimate the effects of reduced caseloads on criminal recidivism. The data, provided by the Iowa Department of Corrections, include probation and court information for 3,254 probationers under supervision during the years 2001-2007.

The study team initially sought to implement RCT in several judicial districts in Colorado (Dataset 4). However, the resources of some jurisdictions did not allow for implementation of a design that required maintenance of reduced caseloads for an extended period of time. Thus, the four largest districts in Colorado with a total of 111,523 probation cases participated in the regression discontinuity design study (RDD). The state Division of Probation Services provided the study team with a multi-year cohort of probation data, from 1997-2007. Outcome data were derived from court filings data provided by the Colorado State Court Administrator's Office.

All probation officers in the study completed audio tapes of their supervision sessions and completed a battery of four survey instruments at two points in the study, with a year interval in between (Dataset 5). The tapes were analyzed by independent raters trained in the three different recognized systems for behavioral rating of officer skills: (1) Motivational Interviewing Therapeutic Integrity-2 (MITI-2); (2) The Dual Role Inventory-Revised (DRI-R); and (3) The Strategic Initiative for Community Supervision (STICS). These rating systems focus respectively on Motivational Interviewing (MI) skills, balance in required roles and the content - criminogenic needs versus non-criminogenic needs - focus of the supervision session. The survey data were derived from four survey tools (Likert Organizational Climate Scale; Intrinsic Motivation Inventory; Probation and Parole Strategies Questionnaire; and, the Employee Satisfaction Survey) previously used in probation survey studies. Measures of relevant constructs were established using data from 72 probation officers including 18 officers in Iowa, 22 officers in Oklahoma, and 32 officers in Colorado at Wave 1. At Wave 2, data were available from the same 18 officers in Iowa, 15 of the original 22 officers in Oklahoma, and 29 of the 32 officers in Colorado.

The study team screened over two dozen sites to determine their eligibility and appropriateness for the study. Sites were first screened via interviews and agency documentation for general eligibility. Critical criteria for eligibility in the study were a sufficient sample size of probationers and probation officers; adequate progress made in implementation of evidence-based practices (EBP); minimum data availability; and general willingness to participate. Sites that met these initial criteria were invited to participate in a more rigorous screening process. There were three sites that met this criteria: Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado.

The Oklahoma City Evidence-Based Practices Pre-Implementation Data (Dataset 1) are comprised of 43,688 probation cases and the Oklahoma City Evidence-Based Practices Post-Implementation Data (Dataset 2) are comprised of 5,905 probation cases. The Polk County, Iowa Data (Dataset 3) are comprised of an overall sample size of 3,254 cases. A total of 111,523 cases from the four largest districts in Colorado are included in the study (Dataset 4). In the Evidence-Based Practices Probation Officer Data (Dataset 5), a convenient sample of 72 probation officers was drawn from three state probation systems using a sampling frame that split the total sample between officers with small caseloads and officers with regular caseload sizes.

Longitudinal: Cohort / Event-based

Datasets 1 and 2: All probation cases in Oklahoma City, Oklahoma between 2001 and 2010.

Dataset 3: All probation cases in Polk County, Iowa between 2001 and 2007.

Dataset 4: All probation cases in Colorado between 1997 and 2007

Dataset 5: All probation officers in Oklahoma City, Oklahoma, Polk County, Iowa, and Colorado between 2007 and 2010.

Datasets 1-4: probation case

Dataset 5: probation officer

In Dataset 3 the Iowa Department of Corrections provided a matched file of court filing and sentence data for all active probation cases for 2000-2007.

In Datasets 1 and 2 the Oklahoma Department of Corrections provided probation data for all supervision cases directly before and during the reduced caseload intervention. Criminal history data was obtained through a query of arrest records from the Oklahoma State Bureau of Investigation.

In Dataset 5 the survey data was derived from four survey tools the Likert Organizational Climate Scale, Intrinsic Motivation Inventory, Probation and Parole Strategies Questionnaire, and the Employee Satisfaction Survey. Trained raters reviewed and scored audiotapes of officers' supervision sessions with probationers on their caseloads.

In Dataset 4 the Colorado State Division of Probation Services provided the study team with a ten-year cohort of probation data for all judicial districts in Colorado. The study team selected the districts with sufficient sample to support the study.

administrative records data

observational data

survey data

experimental data

The Oklahoma City Evidence-Based Practices Pre-Implementation Data (Dataset 1) contain a total of 14 variables including probation supervision information and demographic characteristics. Probation supervision information includes Level of Service Inventory (LSI) Scores, time on supervision, reasons for termination of probation, and failure of probation. Demographic characteristics include employment, sex, education, prior convictions, race, and whether or not probationer had any substance use or abuse.

The Oklahoma City Evidence-Based Practices Post-Implementation Data (Dataset 2) contain a total of 110 variables including demographic characteristics, history of alcohol or drug abuse, criminal records, and probation information. Demographic characteristics include sex, employment, education, age, and race. History of alcohol or drug abuse measured whether or not probationer abused drugs and alcohol and the severity of abuse. Criminal record information includes number of prior arrests, incarcerations, and probations. Probation information includes whether or not probationer had any probation violations or new convictions, date of arrest offense, number and rate of contacts probationer had with probation officers and probation related services, level and status of mental health, alcohol and drug treatment for probationer.

The Polk County, Iowa Data (Dataset 3) contain a total of 327 variables including demographic characteristics, criminal records, and probation information. Demographic characteristics include sex, race, ethnicity, marital status, education, employment, age, and number of dependents. Criminal record information includes prior juvenile convictions, prior probation revocation, type of prior convictions and year of conviction, number and type of prior offenses. Probation information includes start and end dates of supervision, age at start of supervision, date to first arrest, whether there was a revocation of probation and reason for revocation, reason for change in supervision status, re-arrest charge description and type, number and place of contacts probationer had with probation officers, the Iowa Risk score, which determines the supervision intensity, new arrest charge, whether treatment was needed and completed, type of treatment, risk assessment dates, intervention dates, descriptions, categories, settings, length, and reason for termination.

The Colorado Data (Dataset 4) contain 119 variables including demographic characteristics, probation information, treatment information, and arrest data. Demographic characteristics include age at beginning of supervision, race, sex, date supervision started, number of days under supervision, and whether termination codes were found in probation term. Probation information includes number of days first revocation code found during supervision term, revocation codes found in probation term, number of assessments found in probation term, number of probation officer records found for probation term, supervision level found for probation term, number and rate of collateral contacts, general casework, face to face contacts, no show appointments, victim contacts, office visits, employment verification, domestic violence screenings, curfew checks, home visits, and parent or guardian contact within supervision term. Treatment information includes whether urinalysis treatment records were found, number of urinalysis treatment records found, any treatment records with description found, and number of treatment records found. Arrest data includes number of days between arrest and beginning of supervision, arrest offense, and categorization of arrest offense.

The Evidence-Based Practices Probation Officer Data (Dataset 5) contain a total of 61 variables including data collection site, years of education, and a number of probation officer attitudinal and casework skill variables.

Not applicable

The Evidence-Based Practices Probation Officer Data (Dataset 5) contain survey measures, evidence-based practice (EBP) skill measures, measures of officer supervision session content, and process/outcome measures.

Survey measures include the Intrinsic Motivation Inventory (IMI), the Enforcing Compliance subscale of the IMI, the Probation and Parole Strategies Questionnaire survey (PPSQ), an organizational climate measure, a positive attitude score derived by combining each officer's total scores on the Likert Organizational Climate Scale (LOCS) and the Employment Satisfaction Survey (ESS), as well as law enforcement, case worker, and resource broker subscale orientation measures of the PPSQ.

EBP skill measures include relative Skill Balance (SKB), a global rating composite, a Motivational Interviewing Therapeutic Integrity (MITI) average composite behavioral rating, an Officer Interactive Skills (OIS) measure, a Quality Contact Standards (QCS) score, an officer demeanor score from the Department and Manner of Being subscale of the QCS, an assessment and planning skills score from the Assessment and Planning subscale of the QCS, an engagement in treatment referral score from the Referral for Treatment and Services subscale of the QCS, an enforcement of sanctions score from the Enforcement of Sanctions and Ground Rules QCS subscale, an overuse of questions measure, the Dual Role Inventory-Revised score, and a standardized summary measure of officer skills.

Measures of officer supervision session content that emerged in the training and study for the Canadian Strategic Training Initiative in Community Supervision (STICS) include percentage of supervision session time spent discussing a criminogenic need and percentage of time an officer spends discussing non-criminogenic needs in their taped supervision session. Additional measures of officer supervision session content include the percentage of time in supervision session that an officer spends discussing terms and conditions of supervision, duration of each officer's taped supervision session (measured in minutes), a composite skill and supervision measure, and a hybrid skill measure and supervision content score.

Process/outcome measures include Officer Change Talk (a performance score to assess each officer's influence on the offender), Preparatory Change Talk, and Commitment Change Talk.



2014-02-14 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Standardized missing values.
  • Checked for undocumented or out-of-range codes.


  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.

  • The citation of this study may have changed due to the new version control system that has been implemented.
NACJD logo

This dataset is maintained and distributed by the National Archive of Criminal Justice Data (NACJD), the criminal justice archive within ICPSR. NACJD is primarily sponsored by three agencies within the U.S. Department of Justice: the Bureau of Justice Statistics, the National Institute of Justice, and the Office of Juvenile Justice and Delinquency Prevention.