The purpose of this study is to provide research examining juvenile probation officer adherence to the Youth Level of Service/Case Management Inventory (YLS/CMI) juvenile risk/need assessment (RNA) tool, several years after its implementation in Pennsylvania.
The study team targeted officers from a range of roles and ranks in each county for
interviews, using systematic sampling strategies. For smaller counties (D and
E), the study team targeted all officers. In larger counties (A, B and C) the study team purposively
selected officers from relevant units, to ensure a representation of role types
(e.g. line officers, supervisors, and chief level staff). The study team also selected
randomly where they need to choose among multiple eligible officers within an operational
unite/role type. While in Counties A and B the study team were able to select from all
relevant operational subunits, in County C the study team could only focus on a subset of
operational units because of the size and complexity of the office. This made their
sampling in this county less representative.
YLS/CMI Example Interviews:
The study team sought to target all staff from our core interviews who were direct
users of the YLS/CMI (i.e. they had caseloads, and routinely completed the
YLS/CMI and/or made decisions based on it). As noted, The study team recruited one
observation subject to our target list also, to supplement numbers in County C. Once The study team spoke them, they asked them to talk through a couple of recent examples of YLS/CMIs. This was a convenience sampling approach, but emphasized a focus on recent cases. For juvenile probation officers with supervision caseloads The study team typically sought an example of a case they had recently begun supervising (where they may not have personally conducted the YLS/CMI, but had the completion information available), and an example of a YLS/CMI that they had recently completed themselves (usually an update or case-closing YLS/CMI). For intake or investigations probation officers (who didn't supervise probationers), the study team usually asked for two recently completed YLS/CMIs, which tended to be the initial YLS/CMIs conducted on youth at the beginning of their case.
Observations were targeted at officers when they were meeting with
clients, for example through assessment meetings (e.g. typically at intake), supervision
contacts or court appearances. The study team also sought to steer our observations to
achieve a variety of activities and probation officers. Te tended to work with managers
and administrators to help schedule time with officers at times when
researchers were visiting or available, though this process evolved differently
in each county, and could also involve direct contacts with probation officers
to schedule interviews. In one county, (County B) the study team provided administrators
with target lists of officers to help guide them, to ensure a range of roles. Inevitably,
given our reliance on convenience and the involvement of office managers and
administrators the sampling of observations was influenced by the decisions of
office staff, and could have been shaped to present a particular appearance to
State Reform Leader Interviews:
The study team began by targeting key individuals who the study team knew had been influential leaders
in YLS/CMI implementation and related reforms. Our target list was further expanded
based on the suggestions of research subjects, effectively deploying a
snowballing strategy. Subjects included administrators, probation officers of
varied ranks, and consultants.
The survey targeted all juvenile probation officers (of all ranks) in
all 67 county officers of Pennsylvania. This was 1322 officers.
Juvenile probation officers working within PA. Data collected also relates to some of their clients. State reform leaders.
Juvenile probation officers.
Response rates are described below according to the data collection
86 interviews were conducted across the five case study counties, out
of a target sample of 93 probation staff. This figure includes two field substitutions
of the original target list, made with suitable alternatives for unavailable
target subjects--one of whom had left the organization and one who was
unavailable for interview. This amounts to a response rate of 92 percent
(disregarding substitutions). Across individual counties, the response rate
varied between 88 and 100 percent. Of the seven target subjects not interviewed
(and not substituted), one declined, one was unavailable because of military
service, one left the organization during the fieldwork period, and four were
not interviewed because of other scheduling challenges.
YLS/CMI Example Interviews:
A total of 38 probation officers were interviewed from a target list of
45. This list was based on YLS/CMI-using officers identified in the core
interviews (though one was recruited from an observation episode to boost
numbers in County C). This was an effective response rate of 84 percent, though
the rate across counties between 70 and 100 percent. One eligible subject from
the core interviews was overlooked and not included in the target list of 45.
Of those on the target list who were not interviewed, two declined and five
were ultimately not responsive to follow-up scheduling for interview. Through the
38 interviews conducted, a total of 81 separate YLS/CMI events were discussed
(typically two per officer).
Observations were based on convenience sampling principles that did not
formally target a list of subjects. 74 subjects were observed. Only one
directly declined (to the researcher) to be observed. This produced
approximately 198 hours of direct observations of probation officers at work,
evenly split across counties (i.e. approximately 40 hours in each).
State Reform Leader Interviews:
All 12 subjects targeted were interviewed (i.e. 100 percent response
Of 1322 juvenile probation officers in Pennsylvania targeted for the
survey, a total of 492 officers completed the survey and indicated their county
affiliation (15 additional excluded cases completed the survey without
indicating their county affiliation, preventing their inclusion in analysis).
This represents a response rate of 37 percent of officers across all 67 PA
counties. If we count officers in only the 56 counties (1223 officers) that
ultimately participated, this would represent a response rate of 40 percent.
Responses for case study counties A to D ranged from 55% to 88%. Responses from
County E were not sufficient for separate analysis without risking deductive
The survey used an adaptation of the "organizational
climate" scale, based on work by Taxman et al. 2007