A Multi-site Comparison of Risk Assessments within the Juvenile Justice System, 2007-2013 [United States] (ICPSR 34934)
Version Date: Mar 2, 2018 View help for published
Principal Investigator(s): View help for Principal Investigator(s)
Christopher Baird, National Council on Crime and Delinquency;
Kristen Johnson, National Council on Crime and Delinquency;
Theresa Healy, National Council on Crime and Delinquency
https://doi.org/10.3886/ICPSR34934.v1
Version V1
Summary View help for Summary
These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.
This study examined the validity, reliability, equity and cost of nine juvenile justice risk assessment instruments. It was designed to provide a comprehensive examination of how several risk assessments perform in practice. This study posed the following questions:
- Is each risk assessment instrument sufficiently reliable (i.e., inter-rater reliability) to ensure that decisions regarding level of risk and identified service needs are consistent across the organization?
- What specific risk assessment items are associated with less reliability? What items are rated reliably by staff?
- Is each risk assessment instrument valid? Specifically, what degree of discrimination is attained between assigned risk levels? Could the instrument be improved by adding or deleting specific factors and/or altering cut-off scores?
- Is each risk assessment instrument valid for population subgroups: White/Caucasian, Black/African American, Hispanic/Latino, females, probationers, and youth in aftercare status? Could equity be improved by adding or deleting specific factors or altering cut-off scores?
- What costs are associated with each risk assessment instrument?
The study collection includes 31 SPSS data files
- all_jais_combined.sav (n=1,141; 6 variables)
- ar_fire_final_file_ojjdp-ICPSR.sav (n=119; 205 variables)
- AR_yls_irr_FINAL-ICPSR.sav (n=155; 136 variables)
- azaoc_FINALFILE-ICPSR.sav (n=7,589; 438 variables)
- AZAOC_irr_FINAL-ICPSR.sav (n=458; 101 variables)
- AZDJC_FINAL_FILE-ICPSR.sav (n=1,265; 1,290 variables)
- AZDJC_irr_FINAL-ICPSR.sav (n=55; 120 variables)
- COMMITMENT_FINAL_SAMPLE2-ICPSR.sav (n=11,154; 719 variables)
- FinalDJJReleasesWithRecid_BothYears2-ICPSR.sav (n=90,818; 31 variables)
- FIRE_NE_COMM_FINAL_FILE_OJJDP-ICPSR.sav (n=597; 174 variables)
- fire_ne_probation_final-ICPSR.sav (n=1,077; 237 variables)
- FL_irr_FINAL-ICPSR.sav (n=519; 140 variables)
- GA_irr_FINAL-ICPSR.sav (n=509; 263 variables)
- gafire_boyscommunityALL_FINAL-ICPSR.sav (n=5,009; 781 variables)
- gafire_communityALLforretrofit2-ICPSR.sav (n=6,943; 666 variables)
- gafire_finalsampforanalysis_all-ICPSR.sav (n=7,412; 642 variables)
- gafire_finalsampforanalysis_girls-ICPSR.sav (n=2,005; 768 variables)
- jais_boys_wk_1-ICPSR.sav (n=1,989; 484 variables)
- jais_girls_wk_1-ICPSR.sav (n=745; 484 variables)
- NE_irr_FINAL-ICPSR.sav (n=727; 160 variables)
- OR_irr_FINAL-ICPSR.sav (n=477; 112 variables)
- ORE_FIRE_final-ICPSR.sav (n=12,370; 340 variables)
- PROBATION_FINAL_BOYS_ALL-ICPSR.sav (n=20,621; 837 variables)
- PROBATION_FINAL_GIRLS_ALL-ICPSR.sav (n=6,748; 849 variables)
- va_boyssample-ICPSR.sav (n=1,106; 1,273 variables)
- va_final_sample_fullscreen-ICPSR.sav (n=1,439; 1,237 variables)
- va_girlssample-ICPSR.sav (n=333; 1,256 variables)
- VA_irr_expert_FINAL-ICPSR.sav (n=10; 308 variables)
- VA_irr_worker_FINAL-ICPSR.sav (n=685; 308 variables)
- vafinalsample-ICPSR.sav (n=1,919; 1,200 variables)
- workersurveyfinal-ICPSR.sav (n=400; 69 variables)
Citation View help for Citation
Export Citation:
Funding View help for Funding
Subject Terms View help for Subject Terms
Geographic Coverage View help for Geographic Coverage
Smallest Geographic Unit View help for Smallest Geographic Unit
County
Restrictions View help for Restrictions
Access to these data is restricted. Users interested in obtaining these data must complete a Restricted Data Use Agreement, specify the reasons for the request, and obtain IRB approval or notice of exemption for their research.
Distributor(s) View help for Distributor(s)
Time Period(s) View help for Time Period(s)
Date of Collection View help for Date of Collection
Data Collection Notes View help for Data Collection Notes
-
These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.
Study Purpose View help for Study Purpose
The goal was to provide an objective study of the validity, reliability, and equity of different approaches to risk assessment and to review methods currently used to evaluate validity and reliability. In addition, the study sought to identify each instrument's capacity to estimate risk across different race/ethnicity groups and to identify basic cost parameters about each of the risk assessment tools.
Study Design View help for Study Design
The study design measured inter-rater reliability, validity and equity. Inter-rater reliability of risk assessment was measured by asking a sample of officers/caseworkers to review case files for ten youth, observe a videotaped interview of each youth, and score a risk assessment (or risk/needs assessment) for each youth. Validity and Equity was assessed by testing the predictive validity and equity of the risk assessments involved sampling a cohort of youth on probation or released from a facility (i.e., post-commitment). Recidivism was tracked over a 12-month follow-up period for all sites but one (where only nine months of outcomes were available). Outcome measures were obtained from agency databases and include subsequent arrests, subsequent adjudications, and subsequent placement in a juvenile facility.
An advisory board consisting of researchers, a former head of a juvenile corrections department, and purveyors of the various risk assessment instruments examined in the study helped oversee study design and completion. The advisory board met seven times during the course of the project to review and approve all proposed methods of data collection and analysis, all materials used to conduct reliability testing, and all findings and results.
Sample View help for Sample
Participants included juvenile justice agencies that had implemented risk/needs assessments during the preceding 10 to 12 years. They represent the range of agencies that use risk/needs assessments: county probation, state probation, and state juvenile justice systems responsible for incarcerated youth and those in aftercare.
Time Method View help for Time Method
Universe View help for Universe
Youth involved in the juvenile justice system in Arizona, Arkansas, Florida, Georgia, Nebraska, Oregon, Virginia and Solano County, California.
Unit(s) of Observation View help for Unit(s) of Observation
Data Type(s) View help for Data Type(s)
Mode of Data Collection View help for Mode of Data Collection
Description of Variables View help for Description of Variables
Data file all_jais_combined.sav (n=1,141; 6 variables) contains an ID variable, dichotomous variables related to assessments, and a risk level variable.
Data file ar_fire_final_file_ojjdp-ICPSR.sav (n=119; 205 variables) contains variables pertaining to prior and current offenses, family circumstances, parenting, education, employment, peer relations, substance abuse, leisure/recreation, personality/behavior, and attitudes/orientation.
Data file AR_yls_irr_FINAL-ICPSR.sav (n=155; 136 variables) contains variables related to prior offenses, behavior, substance use, attitudes, and parents.
Data file azaoc_FINALFILE-ICPSR.sav (n=7,589; 438 variables) contains variables related to risk assessment and criminal complaints, petitions, and dispositions.
Data file AZAOC_irr_FINAL-ICPSR.sav (n=458; 101 variables) contains variables related to complaints, family relationships, drug use, school attendance, behavior, friends, and running away.
Data file AZDJC_FINAL_FILE-ICPSR.sav (n=1,265; 1,290 variables) contains variables related to maniuplative behavior, empathy, respect, respect, and accountability. There are also variables regarding conflicts in the home, education, community activities, and parents. Additional variables relate to suicide, employment, drug use, and exposure to violence.
Data file AZDJC_irr_FINAL-ICPSR.sav (n=55; 120 variables) contains variables relate to family conflict, community involvement, family history of drug use, incarceration history of family members, and history of violent behavior.
Data file COMMITMENT_FINAL_SAMPLE2-ICPSR.sav (n=11,154; 719 variables) contains variables related to offenses, confinement, school, recreational activities, employment, family relations, drug use, exposure to violence, mental health, and behavior.
Data file FinalDJJReleasesWithRecid_BothYears2-ICPSR.sav (n=90,818; 31 variables) contains variables related to prison admission and delinquency.
Data file FIRE_NE_COMM_FINAL_FILE_OJJDP-ICPSR.sav (n=597; 174 variables) contains variables pertaining to attributes of juvenile offenders and their situations relevant to decisions regarding level of service, supervision, and programming. They include assessment and summary of risks and needs such as prior and current offenses , peer relations, and substance abuse.
Data file fire_ne_probation_final-ICPSR.sav (n=1,077; 237 variables) contains variables related to arrests, peers, substance abuse, family, education, and behavior.
Data file FL_irr_FINAL-ICPSR.sav (n=519; 140 variables) contains variables related to mental health history, family imprisonment history, substance use history, and violence history.
Data file GA_irr_FINAL-ICPSR.sav (n=509; 263 variables) contains variables related to usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.
Data file gafire_boyscommunityALL_FINAL-ICPSR.sav (n=5,009; 781 variables) contains variables related to arrests, sentencing, adjudications, usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.
Data file gafire_communityALLforretrofit2-ICPSR.sav (n=6,943; 666 variables) contains variables related to arrests, adjudication, usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.
Data file gafire_finalsampforanalysis_all-ICPSR.sav (n=7,412; 642 variables) contains variables pertaining to arrests, adjudication, sentencing, social activities, criminal associates, behaviors and attitudes, substance use, promiscuity, school, and family.
Data file gafire_finalsampforanalysis_girls-ICPSR.sav (n=2,005; 768 variables) contains variables related to arrests, adjudication, sentencing, usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.
Data files jais_boys_wk_1-ICPSR.sav (n=1,989; 484 variables) and jais_girls_wk_1-ICPSR.sav (n=745; 484 variables) contain variables related to offenses, schools, peers, substance use, behavior, and attitudes.
Data file NE_irr_FINAL-ICPSR.sav (n=727; 160 variables) contains variables related to school, peers, behavior, and family.
Data file OR_irr_FINAL-ICPSR.sav (n=477; 112 variables) contains variables related to school, relationships, behavior, family, substance use, and attitudes.
Data file ORE_FIRE_final-ICPSR.sav (n=12,370; 340 variables) contains variables pertaining to demographics such as language and ethnicity, school issues, peer relationships, behavior issues, family functioning, substance use, attitudes, values, and beliefs, and mental health indicators.
Data files PROBATION_FINAL_BOYS_ALL-ICPSR.sav (n=20,621; 837 variables) and PROBATION_FINAL_GIRLS_ALL-ICPSR.sav (n=6,748; 849 variables) contain variables related to criminal history, school, employment, relationships, family, substance use, behaviors, and attitudes.
Data files va_boyssample-ICPSR.sav (n=1,106; 1,273 variables), va_final_sample_fullscreen-ICPSR.sav (n=1,439; 1,237 variables), va_girlssample-ICPSR.sav (n=333; 1,256 variables), and vafinalsample-ICPSR.sav (n=1,919; 1,200 variables) contain variables related to legal history, family, school, community and peers, alcohol and drugs, mental health, aggression, attitudes, skills, and employment and free time.
Data files VA_irr_expert_FINAL-ICPSR.sav (n=10; 308 variables) and VA_irr_worker_FINAL-ICPSR.sav (n=685; 308 variables) contain variables related to family, substance use, mental health, victimization, and violence.
Data file workersurveyfinal-ICPSR.sav (n=400; 69 variables) contains variables related to the opinions of juvenile justice staff who participated in the study. Respondents were asked their opinions about the risk assessment instrument design, how often they agreed with the risk level assigned by the risk assessment instrument, and whether they believed the risk assessment instrument was effective. In addition, staff were asked to provide feedback on the training they received since the assessment instrument was implemented.
Response Rates View help for Response Rates
Not applicable
Presence of Common Scales View help for Presence of Common Scales
The following risk assessment instruments were used in the study:
- Georgia Comprehensive Risk/Needs Assessment (CRN)
- The Oregon Juvenile Crime Prevention (JCP) Risk Assessment
- Girls Link risk assessment instrument
- Positive Achievement Change Tool (PACT)
- Youth Assessment and Screening Instrument (YASI)
- Youth Level of Service/Case Management Inventory (YLS/CMI)
- The Arizona Department of Juvenile Corrections Dynamic Risk Instrument (DRI)
- The Arizona Juvenile Risk and Needs Assessment.
- The Juvenile Sanctions Center (JSC) risk assessment
Notes
These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.
The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.
One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.