The purpose of the study was to determine whether risk assessment tools maintain their predictive accuracy across different developmental periods.
The study was a secondary data analysis of risk assessment tools in their accuracy for predicting re-offending behavior at different periods of development. This included a search of all published and unpublished datasets which met the following criteria: (a) an evidence-based risk assessment tool, (b) studied an offender (corrections, pretrial detention, or probation) or forensic psychiatric population (civil psychiatric samples were not included), and (c) included a measure of re-offending in the community using self-report or official records. The investigators contacted original authors of these studies and requested their de-identified data sets. Datasets were combined by risk tool to examine recidivism outcomes.
Follow-up periods for tracking re-offending spanned 12 to 85 months. The unit of analysis was each case (e.g., young or adult offender or forensic psychiatric patient). Ten studies were used, located in the Netherlands, Western and Eastern Canada, North Carolina, and Connecticut. While the time frame for data collection spanned from 1993 to 2006, most of the studies selected their sample based on a particular time frame (for example, all individuals released from a forensic psychiatric facility from 1993 to 1999).
The investigators combined data sets that involved the same risk assessment tools to examine the predictive accuracy of each instrument for any re-offending (includes all types of offenses) and for violent re-offending specifically. The datasets were combined according to five scales that examined youth or adult risk assessment.
The adolescent risk assessment tools included the North Carolina Assessment of Risk (NCAR, Dataset 1), the Youth Level of Service/Case Management Inventory (YLS/CMI, Dataset 2), and the Structured Assessment of Violence Risk for Youth (SAVRY, Dataset 3).
The adult risk assessment tools included the Historical Clinical Risk Management-20 items (HCR-20, Dataset 4) and the violence Risk Appraisal Guide (VRAG, Dataset 5).
Each of these tools are valid risk assessments that are completed using clinical rating scales for each item based on a combination of information obtained by an interview with the offenders and correctional information. All studies gathered data using these procedures where live cases were assessed by the researchers or the researchers conducted the assessments completely based on archival correctional/psychiatric file reviews.
The initial sample contained 10,617 risk assessment evaluations, 2,353 adult cases and 8,264 adolescent cases. The distribution of these evaluations among the risk assessment tools were as follows; North Carolina Assessment of Risk (NCAR) = 6,751, Structured Assessment of Violence Risk for Youth (SAVRY) = 972, Youth Level of Service/Case Management Inventory (YLS/CMI) = 541, Violence Risk Appraisal Guide (VRAG) = 1,717, Historical Clinical Risk Management (HCR-20) = 636. The sample included cases from 10 studies from the Netherlands, Western and Eastern Canada, North Carolina, and Connecticut.
To determine whether these tools were as accurate at predicting re-offending during early adulthood (age 18 to 25 years) as predicting re-offending during adolescence (age 12-17 years; youth tools only) or in later adulthood (older than 25 years, adult tools only), the researchers excluded cases that did not have a follow-up period spanning into early adulthood. Thus, the final sample included only cases that had been assessed (or released from an institution) prior or equal to age 23 (for adult tools), or that were tracked until age 19 or older (for youth tools). This procedure resulted in at least a one year follow-up into early adulthood for every case. As such, a considerable number of cases were dropped from many of the datasets. This resulted in a final sample of 4,031 cases. The final sample size for each risk assessment tool was as follows: NCAR = 2,129, SAVRY = 786, YLS/CMI = 400, VRAG = 589, HCR-20 = 127.
All published and unpublished datasets that; (a) had an evidence based risk assessment tool, (b) studied an offender or forensic psychiatric population, and (c) included a measure of re-offending in the community, between 1993 and 2006.
Data was collected through review of administrative and archival records, interviews and other collateral information. The following studies were used by researchers:
- North Carolina Department of Juvenile Justice (unpublished data),
- De Vogel, de Ruiter, Hildebrand, Bos and Va de Ven (2004),
- Douglas (1999),
- Harris, Rice, and Cormier (2002),
- Lodewijks, de Ruiter, and Doreleijers (2008),
- Loza, Villeneuve, and Loza-Fanous (2002),
- Rowe (2002),
- Vincent, Quinlan, Nitschelm, and Ogloff (2003),
- Vincent, Chapman, and Cook (under review), and
- Welsh, Schmidt, McKinnon, Chattha, and Meyers (2008).
Dataset 1 variables included offender demographics (age and race), admission information (offense, date, and adjudication), NCAR score, follow up information (age, time at risk, and date), and recidivism information (violence, days to re-offense, and age at recidivism).
Dataset 2 variables included offender demographics (age, gender, and ethnicity), YLS scores, re-offense information (number, violence, age, and months to re-offense), and follow up information (length of period, time at risk, and age).
Dataset 3 variables included offender demographics (age, race and gender), SAVRY scores, re-offense information (number, age, and violence), follow up information (age and days to re-offense), and recidivism (age and violence).
Dataset 4 variables included offender demographics (age, race, gender, and birth place), admission information (age, date, and charges), discharge information (age and date), follow up information (date, age, and time at risk), recidivism information (date, charges, and violence), and HCR-20 score.
Dataset 5 variables included offender demographics (gender, race, and age), admission (age and date), VRAG scores, follow up information (time at risk, age, and parole status), and recidivism information (age and days to re-offense).
The response rates for studies used in these data were not reported and were not always relevant (some studies used archival data).
- North Carolina Assessment of Risk (NCAR)
- Youth Level of Service/Case Management Inventory (YLS/CMI)
- Structured Assessment of Violence Risk for Youth (SAVRY)
- Historical Clinical Risk Management (HCR)
- Violence Risk Appraisal Guide (VRAG)