Probation officers' Compliance with the Youth Level of Service/Case Management Inventory (YLS/CMI): A Multi-level Study of Post-implementation Practice across Pennsylvania counties, 2015-2018 (ICPSR 37201)

Time Period(s) View help for Time Period(s)

Date of Collection View help for Date of Collection

Study Purpose View help for Study Purpose

The purpose of this study is to provide research examining juvenile probation officer adherence to the Youth Level of Service/Case Management Inventory (YLS/CMI) juvenile risk/need assessment (RNA) tool, several years after its implementation in Pennsylvania.

Sample View help for Sample

Core interviews:

The study team targeted officers from a range of roles and ranks in each county for interviews, using systematic sampling strategies. For smaller counties (D and E), the study team targeted all officers. In larger counties (A, B and C) the study team purposively selected officers from relevant units, to ensure a representation of role types (e.g. line officers, supervisors, and chief level staff). The study team also selected randomly where they need to choose among multiple eligible officers within an operational unite/role type. While in Counties A and B the study team were able to select from all relevant operational subunits, in County C the study team could only focus on a subset of operational units because of the size and complexity of the office. This made their sampling in this county less representative.

YLS/CMI Example Interviews:

The study team sought to target all staff from our core interviews who were direct users of the YLS/CMI (i.e. they had caseloads, and routinely completed the YLS/CMI and/or made decisions based on it). As noted, The study team recruited one observation subject to our target list also, to supplement numbers in County C. Once The study team spoke them, they asked them to talk through a couple of recent examples of YLS/CMIs. This was a convenience sampling approach, but emphasized a focus on recent cases. For juvenile probation officers with supervision caseloads The study team typically sought an example of a case they had recently begun supervising (where they may not have personally conducted the YLS/CMI, but had the completion information available), and an example of a YLS/CMI that they had recently completed themselves (usually an update or case-closing YLS/CMI). For intake or investigations probation officers (who didn't supervise probationers), the study team usually asked for two recently completed YLS/CMIs, which tended to be the initial YLS/CMIs conducted on youth at the beginning of their case.

Observational Data:

Observations were targeted at officers when they were meeting with clients, for example through assessment meetings (e.g. typically at intake), supervision contacts or court appearances. The study team also sought to steer our observations to achieve a variety of activities and probation officers. Te tended to work with managers and administrators to help schedule time with officers at times when researchers were visiting or available, though this process evolved differently in each county, and could also involve direct contacts with probation officers to schedule interviews. In one county, (County B) the study team provided administrators with target lists of officers to help guide them, to ensure a range of roles. Inevitably, given our reliance on convenience and the involvement of office managers and administrators the sampling of observations was influenced by the decisions of office staff, and could have been shaped to present a particular appearance to researchers.

State Reform Leader Interviews:

The study team began by targeting key individuals who the study team knew had been influential leaders in YLS/CMI implementation and related reforms. Our target list was further expanded based on the suggestions of research subjects, effectively deploying a snowballing strategy. Subjects included administrators, probation officers of varied ranks, and consultants.

Survey:

The survey targeted all juvenile probation officers (of all ranks) in all 67 county officers of Pennsylvania. This was 1322 officers.

Time Method View help for Time Method

Universe View help for Universe

Juvenile probation officers working within PA. Data collected also relates to some of their clients. State reform leaders.

Unit(s) of Observation View help for Unit(s) of Observation

Data Type(s) View help for Data Type(s)

Mode of Data Collection View help for Mode of Data Collection

Response Rates View help for Response Rates

Response rates are described below according to the data collection method:

Core interviews:

86 interviews were conducted across the five case study counties, out of a target sample of 93 probation staff. This figure includes two field substitutions of the original target list, made with suitable alternatives for unavailable target subjects--one of whom had left the organization and one who was unavailable for interview. This amounts to a response rate of 92 percent (disregarding substitutions). Across individual counties, the response rate varied between 88 and 100 percent. Of the seven target subjects not interviewed (and not substituted), one declined, one was unavailable because of military service, one left the organization during the fieldwork period, and four were not interviewed because of other scheduling challenges.

YLS/CMI Example Interviews:

A total of 38 probation officers were interviewed from a target list of 45. This list was based on YLS/CMI-using officers identified in the core interviews (though one was recruited from an observation episode to boost numbers in County C). This was an effective response rate of 84 percent, though the rate across counties between 70 and 100 percent. One eligible subject from the core interviews was overlooked and not included in the target list of 45. Of those on the target list who were not interviewed, two declined and five were ultimately not responsive to follow-up scheduling for interview. Through the 38 interviews conducted, a total of 81 separate YLS/CMI events were discussed (typically two per officer).

Observational Data:

Observations were based on convenience sampling principles that did not formally target a list of subjects. 74 subjects were observed. Only one directly declined (to the researcher) to be observed. This produced approximately 198 hours of direct observations of probation officers at work, evenly split across counties (i.e. approximately 40 hours in each).

State Reform Leader Interviews:

All 12 subjects targeted were interviewed (i.e. 100 percent response rate).

Survey:

Of 1322 juvenile probation officers in Pennsylvania targeted for the survey, a total of 492 officers completed the survey and indicated their county affiliation (15 additional excluded cases completed the survey without indicating their county affiliation, preventing their inclusion in analysis). This represents a response rate of 37 percent of officers across all 67 PA counties. If we count officers in only the 56 counties (1223 officers) that ultimately participated, this would represent a response rate of 40 percent. Responses for case study counties A to D ranged from 55% to 88%. Responses from County E were not sufficient for separate analysis without risking deductive disclosure.

Presence of Common Scales View help for Presence of Common Scales

The survey used an adaptation of the "organizational climate" scale, based on work by Taxman et al. 2007

Original Release Date View help for Original Release Date

2020-02-26

Version History View help for Version History

Weight View help for Weight

Not applicable.