Supplemental Materials

Risk Assessment Matters, but Only When Implemented Well: A Multi-Site Study in Juvenile Probation

by G. M. Vincent et al., 2016, Law and Human Behavior

Supplementary Materials:

Group Demographics Before and After Propensity Score Matching for the First Site in Each Statea

SAVRY Site 1
Pre-Implementation Group / Post-Implementation Group
Before Match
(n = 247) / After Match
(n = 205) / Before Match
(n = 217) / After Match
(n = 205)
Gender
% Male
% Female / 72.9%
27.1% / 70.7%
29.3% / 65.9%
34.1% / 66.8%
33.2%
Race
% African American /Black
% White / 88.7%
10.9% / 86.8%
12.7% / 86.2%
12.9% / 86.8%
12.2%
% Hispanic / 0.4% / 0.5% / 0.0% / 0.0%
% Violent Index Offense / 29.1% / 28.3% / 24.0% / 25.4%
% Nonviolent Felony / 10.9%*** / 12.2% / 23.0%*** / 19.0%
Age at Adjudication / M = 15.06
SD = 1.66 / M = 15.01
SD = 1.69 / M = 15.16
SD = 1.54 / M = 15.01
SD = 1.56
Age at First Offense / M = 13.67
SD = 1.66 / M = 13.73
SD = 1.71 / M = 13.95
SD =1.65 / M = 13.85
SD = 1.65
Prior State JJ Commitment / 5.3% / 5.4% / 3.2% / 3.4%
% Any Violent Priors / 37.7%*** / 26.8% / 18.5%*** / 19.5%
Mean # of Prior Arrests / M = 2.36***
SD = 2.67
Range = 0-17 / M = 1.72*
SD = 1.93
Range = 0-13 / M = 1.23***
SD = 2.07
Range = 0-12 / M = 1.29*
SD = 2.12
Range = 0-12
Any Axis I Diagnosis / 15.4% / 15.6% / 12.0% / 12.7%
Any Axis II Diagnosis / 8.1% / 7.8% / 7.8% / 8.3%
Prior Outpatient Mental Health
Treatment / 17.3% / 17.2% / 21.3% / 21.1%
Prior Outpatient Substance Abuse
Treatment / 0.4% / 0.5% / 0.5% / 0.5%
Any Grade Failure / 25.9% / 27.1% / 29.3% / 29.9%
Regular School Attendance / 70.4% / 71.4% / 66.2% / 65.1%
Good School Performance / 70.5% / 73.1% / 73.9% / 72.8%
Living Arrangement
% Both Parents
% Single Parent
% Relative / 18.2%
68.8%
10.9% / 16.6%
69.8%
11.7% / 15.2%
71.4%
12.0% / 15.6%
70.7%
12.2%
Currently Placed Out of the Home / 1.3% / 1.5% / 3.7% / 3.4%
History of Child Welfare Involvement / 8.1% / 8.7% / 8.3% / 7.3%
SAVRY Site 2
Pre-Implementation Group / Post-Implementation Group
Before Match
(n =161) / After Match
(n = 92) / Before Match
(n =105) / After Match
(n = 92)
Gender
% Male
% Female / 77.0%
23.0% / 75.0%
25.0% / 81.0%
19.0% / 79.3%
20.7%
Race
% African American /Black
% White / 49.1%
47.2% / 55.4%
41.3% / 61.9%
33.3% / 60.9%
38.0%
% Hispanic / 1.2% / 0% / 0% / 0%
Age at adjudication / M = 14.40
SD = 1.94 / M = 14.51
SD = 2.22 / M = 14.51
SD =2.41 / M =14.33
SD =2.51
Age at first offense / M = 12.76
SD = 2.50 / M = 12.91
SD =2.68 / M = 12.88
SD =2.66 / M = 12.73
SD =2.77
% Violent index offense / 36.6%** / 27.2% / 20.0%** / 20.7%
% Any Violent Priors / 21.7% / 22.8% / 25.7% / 22.8%
Mean # of Prior Arrests / M = 2.0
SD = 2.39
Range = 0-11 / M =2.05
SD =2.44
Range =0-11 / M = 2.38
SD =2.96
Range =0-14 / M =2.24
SD =2.85
Range =0-14
Any Axis I Diagnosis / 33.5%*** / 18.5% / 12.4%*** / 13.0%
Any Axis II Diagnosis / 36.6%*** / 21.7% / 13.3%*** / 15.2%
Prior Outpatient Mental Health
Treatment / 36.0%* / 25.9% / 20.7%* / 22.4%
Current Outpatient Mental Health
Treatment / 38.1%*** / 15.5% / 11.8%*** / 12.3%
Current Outpatient Substance Abuse Treatment / 8.2%* / 7.2% / 1.3%* / 1.5%
Any Grade Failure / 26.7% / 20.0% / 13.8% / 17.0%
Regular School Attendance / 58.1% / 61.0% / 60.3% / 55.7%
Good School Performance / 43.3%* / 48.2% / 61.4%* / 56.9%
Living Arrangement
% Both Parents
% Single Parent
% Relative / 16.1%
63.4%
16.8% / 16.3%
62.0%
17.4% / 9.6%
69.2%
13.5% / 9.9%
69.2%
13.2%
History of child welfare involvement / 34.4%* / 31.8% / 20.9%* / 18.9%
SAVRY Site 3
Pre-Implementation Group / Post-Implementation Group
Before Match
(n = 239) / After Match
(n = 117) / Before Match
(n = 130) / After Match
(n =117)
Gender
% Male
% Female / 68.6%
31.4% / 79.5%
20.5% / 76.2%
23.8% / 77.8%
22.2%
Race
% African American/Black
% White / 76.6%
23.0% / 83.8%
16.2% / 82.3%
16.2% / 82.1%
16.2%
% Hispanic / 0.4% / 0% / 1.5% / 1.7%
% Adjudicated FINS / 36.4%*** / 17.1% / 13.1%*** / 13.7%
Age at Adjudication / M = 15.61
SD = 1.19 / M = 15.62
SD = 1.22 / M = 15.47
SD = 1.19 / M = 15.53
SD = 1.21
% Violent Index Offense / 19.2% / 24.8% / 26.9% / 26.5%
Mean # of Prior Arrests / M = 0.61*
SD = 1.19
Range = 0-7 / M = .35
SD = .67
Range = 0-4 / M = .35*
SD = .82
Range = 0-4 / M = .38
SD = .85
Range = 0-4
% Any Violent Priors / 8.8% / 5.1% / 5.4% / 6.0%
Any Axis I Diagnosis / 6.3% / 6.0% / 5.4% / 5.1%
Any Axis II Diagnosis / 7.5% / 6.0% / 4.6% / 4.3%
Prior Outpatient Mental Health Treatment / 22.2%** / 18.8% / 24.6%** / 23.9%
Current Outpatient Mental Health Treatment / 10.9% / 8.5% / 11.5% / 10.3%
Prior Outpatient Substance Abuse Treatment / 5.9%*** / 2.6% / 2.3%*** / 2.6%
Any Grade Failure / 15.5%* / 20.5% / 25.4%* / 25.6%
Regular School Attendance / 52.7%** / 67.5% / 61.5%** / 66.7%
Good School Performance / 15.5%** / 21.4% / 25.4%** / 27.4%
Living Arrangement
% Both Parents
% Single Parent
% Relative / 13.4%**
71.1%**
13.0%** / 10.3%
75.2%
12.8% / 6.9%**
72.3%**
14.6%** / 7.7%
76.1%
13.7%
History of Child Welfare Involvement / 9.2%** / 6.0% / 6.9%** / 6.0%
YLS/CMI Site 1
Pre-Implementation Group / Post-Implementation Group
Before Match
(n = 242) / After Match
(n = 221) / Before Match
(n = 232) / After Match
(n = 221)
Gender
% Male
% Female / 70.7%
29.3% / 70.6%
29.4% / 75.0%
25.0% / 74.7%
25.3%
Race
% African American/Black
% White / 21.5%
59.9% / 21.3%
59.7% / 18.1%
57.8% / 18.6%
57.9%
% Hispanic / 21.1% / 22.2% / 21.6% / 20.8%
Age at Adjudication / M = 15.72
SD = 1.63 / M = 15.70
SD = 1.64 / M = 15.60
SD = 1.57 / M = 15.59
SD = 1.54
Age at First Offense / M = 15.27
SD = 1.85 / M = 15.27
SD = 1.85 / M = 15.32
SD = 1.88 / M = 15.36
SD = 1.86
% Violent Index Offense / 17.8% / 16.7% / 15.1% / 15.4%
Mean # of Prior Arrests / M = .59
SD = 1.15
Range = 0-7 / M = .58
SD = 1.68
Range = 0-7 / M = .56
SD = 1.09
Range = 0-6 / M = .52
SD = 1.02
Range = 0-5
% Any Violent Priors / 9.5% / 8.6% / 9.5% / 8.1%
Any Axis I Diagnosis / 9.9% / 8.1% / 6.1% / 5.9%
Any Axis II Diagnosis / 14.9% / 13.1% / 10.5% / 10.0%
Prior Outpatient Mental Health Treatment / 18.4% / 18.5% / 17.6% / 17.0%
Current Outpatient Mental Health Treatment / 17.9% / 18.1% / 14.5% / 13.8%
Prior Outpatient Substance Abuse Treatment / 7.7% / 7.4% / 5.7% / 6.0%
Current Outpatient Substance Abuse Treatment / 8.2% / 7.4% / 4.4% / 4.6%
Any Grade Failure / 2.8% / 2.4% / 4.9% / 4.6%
Regular School Attendance / 86.4%** / 85.6%* / 76.0%** / 76.5%*
Good School Performance / 87.7% / 88.4% / 84.4% / 84.4%
Living Arrangement
% Both Parents
% Single Parent
% Relative / 37.9%
55.4%
4.2% / 39.3%
54.3%
3.7% / 31.6%
58.4%
2.6% / 33.0%
57.9%
2.7%
History of Child Welfare Involvement / 6.8% / 6.0% / 8.8% / 8.3%
YLS Site 2
Pre-Implementation Group / Post-Implementation Group
Before Match
(n = 186) / After Match
(n = 104) / Before Match
(n = 116) / After Match
(n =104)
Gender
% Male
% Female / 79.0%
21.0% / 78.8%
21.2% / 80.2%
19.8% / 79.8%
20.2%
Race
% African American/Black
% White / 14.0%
82.8% / 10.6%
86.5% / 8.6%
87.9% / 6.7%
89.4%
% Hispanic / 7.0% / 5.8% / 7.8% / 4.8%
Age at Adjudication / M = 15.89
SD = 1.54 / M = 15.58
SD = 1.53 / M = 15.57
SD = 1.79 / M = 15.86
SD = 1.46
Age at First Offense / M = 15.12
SD = 1.65 / M = 15.01*
SD = 1.61 / M = 15.15
SD = 1.86 / M = 15.44*
SD = 1.54
% Violent Index Offense / 18.3% / 18.3% / 16.4% / 16.3%
% Any Violent Priors / 10.2%** / 1.9% / 0.9%** / 1.0%
Mean # of Prior Arrests / M = .85***
SD = 1.74
Range = 0-10 / M = .44
SD = 1.13
Range = 0-6 / M = .17***
SD = .75
Range = 0-6 / M = .19
SD = .79
Range = 0-6
Any Axis I Diagnosis / 23.1% / 19.2% / 23.3% / 17.3%
Any Axis II Diagnosis / 11.3% / 7.7% / 5.2% / 3.8%
Prior Outpatient Mental Health Treatment / 50.3%* / 42.7% / 34.9%* / 34.7%
Current Outpatient Mental Health Treatment / 30.3% / 28.7% / 32.4% / 29.2%
Prior Outpatient Substance Abuse Treatment / 30.1%** / 26.7%* / 12.5%** / 14.0%*
Current Outpatient Substance Abuse Treatment / 16.5% / 17.6% / 24.5% / 26.3%
Good School Performance / 14.5%** / 16.7% / 26.7%** / 25.3%
Living Arrangement
% Both Parents
% Single Parent
% Relative / 30.8%
59.1%
5.7% / 34.3%
56.9%
4.9% / 28.3%
58.4%
7.1% / 31.7%
58.4%
4.0%
Currently Placed Out of the Home / 4.8% / 6.1% / 4.6% / 4.1%
History of Child Welfare Involvement / 24.7%*** / 27.9% / 25.0%*** / 21.2%
YLS Site 3
Pre-Implementation Group / Post-Implementation Group
Before Match
(n = 161) / After Match
(n = 108) / Before Match
(n = 231) / After Match
(n = 108)
Gender
% Male
% Female / 83.9%
16.1% / 81.5%
18.5% / 79.7%
20.3% / 82.4%
17.6%
Race
% African American
% White / 85.1%***
12.4%*** / 81.5%
15.7% / 54.1%***
44.6%*** / 83.3%
14.8%
% Hispanic / 3.1%*** / 4.6% / 33.8%*** / 4.6%
Age at Adjudication / M = 15.88*
SD = 1.56 / M = 16.11
SD = 1.32 / M = 16.27*
SD = 1.59 / M = 16.03
SD = 1.65
Age at First Offense / M = 14.74
SD = 1.84 / M = 14.83
SD = 1.78 / M = 14.85
SD = 1.87 / M = 14.85
SD = 1.88
% Violent Index Offense / 42.2% / 44.4% / 43.3% / 50.9%
% Any Violent Priors / 35.4% / 38.0% / 32.0% / 38.9%
Mean # of Prior Arrests / M = 1.32
SD = 1.60
Range = 0-12 / M = 1.39
SD = 1.67
Range = 0-12 / M = 1.42
SD = 1.60
Range = 0-8 / M = 1.38
SD = 1.55
Range = 0-7
Prior Outpatient Mental Health Treatment / 44.1% / 46.3% / 51.1% / 45.4%
Current Outpatient Mental Health Treatment / 23.0% / 21.3% / 23.4% / 18.5%
Current Outpatient Substance Abuse Treatment / 11.8% / 12.0% / 16.5% / 8.3%
Regular School Attendance / 24.2%** / 28.7% / 31.6%** / 26.9%
Living Arrangement
% Both Parents
% Single Parent
% Relative / 8.9%
74.1%
13.9% / 9.5%
73.3%
14.3% / 10.7%
72.9%
10.2% / 9.5%
76.2%
12.4%
History of Child Welfare Involvement / 64.0% / 62.0% / 63.2% / 64.8%

Note. For SAVRY Site 1, significant differences in pre/post-samples before match: % nonviolent felony, p < .001, phi = 0.16, 95% CI [0.07, 0.25]; mean number of prior arrests, p < .001, d = 0.47, 95% CI [0.70, 1.56]; % any violent priors, p < .001, phi = -0.21, 95% CI [-0.30, -0.13]. Significant differences in pre/post-samples after match: mean # of prior arrests, p = .03, d = 0.21, 95% CI [0.04, 0.83]. For YLS/CMI Site 1, significant differences in pre/post-samples before match: regular school attendance, p = .006, phi = -0.13, 95% CI [-0.23, -0.04]. Significant differences in pre/post-samples after match: regular school attendance,

p = .020, phi = -0.12, 95% CI [-0.21, -0.02]. In SAVRY Site 2, significant differences in pre/post-samples before match: % violent index offense, p = .004, phi = -0.18, 95% CI [-0.29, -0.05]; axis I diagnosis,

p < .001, phi = -0.24, 95% CI [-0.34, -0.13]; axis II diagnosis, p < .001, phi = -0.26, 95% CI [-0.36, -0.15]; prior outpatient mental health treatment, p = .013, phi = -0.16, 95% CI [-0.27, -0.04]; current outpatient mental health treatment, p < .001, phi = -0.29, 95% CI [-0.40, -0.16]; current outpatient substance abuse treatment, p = .035, phi = -0.14, 95% CI [-0.21, -0.04]; good school performance, p = .021, phi = 0.18, 95% CI [0.03, 0.32]; history of child welfare involvement, p = .02, phi = -0.15, 95% CI [-0.27, -0.03]. In SAVRY Site 3, significant differences in pre/post-samples before match: % adjudicated FINS, p < .001,

phi = -0.25, 95% CI [-0.33, -0.16]; mean # of prior arrests, p = .017, d = 0.25, 95% CI [0.05, 0.46]; prior outpatient mental health treatment, p = .002, Cramer’s V = 0.19, 95% CI [0.10, 0.29]; prior outpatient substance abuse treatment, p < .001, Cramer’s V = 0.22, 95% CI [0.13, 0.31]; any grade failure, p =.015, Cramer’s V = 0.15, 95% CI [0.07, 0.25]; regular school attendance, p = .004, Cramer’s V = 0.19, 95% CI [0.12, 0.27]; good school performance, p = .002, Cramer’s V = 0.19, 95% CI [0.10, 0.29]; living arrangement, p = .008, Cramer’s V = 0.19, 95% CI [0.13, 0.29]; history of child welfare involvement,

p = .002, Cramer’s V = 0.18, 95% CI [0.09, 0.28]. For YLS/CMI Site 2, significant differences in pre/post-samples before match: any prior violent offenses, p = .001, phi = -0.18, 95% CI [-0.25, -0.11]; mean # of prior arrests, p < .001, d = 0.51, 95% CI [0.40, 0.97]; prior outpatient mental health treatment; p = .011,

phi = -0.15, 95% CI[-0.27, -0.04]; prior outpatient substance abuse treatment, p = .001, phi = -0.20, 95% CI [-0.30, -0.10; good school performance, p = .008, Cramer’s V = 0.19, 95% CI [0.08, 0.30]; history of child welfare involvement, p < .001, Cramer’s V = 0.28, 95% CI [0.20, 0.36]. For YLS/CMI Site 2, significant differences in pre/post-samples after match: age at first offense, p = .049, d = -0.27, 95% CI

[-0.86, -0.00]; prior outpatient substance abuse treatment, p = .025, phi = -0.16, 95% CI [-0.29, -0.02]. For YLS/CMI Site 3, significant differences in pre/post-samples before match: race, p < .001,

Cramer’s V = 0.36, 95% CI [0.29, 0.44]; ethnicity: p < .001, phi = 0.37, 95% CI [.0.31, 0.44]; age at adjudication: p =.019, d = -0.24, 95% CI [-0.70, -0.06]; regular school attendance: p = .002,

Cramer’s V = 0.18, 95% CI [0.10, 0.28].

*p < .05. **p < .01. ***p < .001

Supplementary Materials:

Recidivism Rates by Site

New Petitions / New Adjudications
Pre-Implementation / Post-Implementation / Pre-Implementation / Post-Implementation
Rate % / SE / Rate % / SE / Rate % / SE / Rate % / SE
SAVRY Site 1
Any / 42 / .04 / 46 / .04 / 28 / .03 / 37*** / .03
Violent / 20 / .03 / 20 / .03 / 13 / .02 / 15 / .03
Non-violent / 27 / .03 / 34 / .03 / 15 / .03 / 23** / .03
Violations / 10 / .02 / 15 / .03 / 5 / .02 / 11* / .02
SAVRY Site 2
Any / 52 / .06 / 60 / .06 / 43 / .05 / 51 / .06
Violent / 16 / .04 / 13 / .04 / 9 / .03 / 6 / .03
Non-violent / 38 / .05 / 49 / .05 / 24 / .05 / 28 / .05
Violations / 42 / .06 / 48 / .06 / 38 / .05 / 45 / .06
SAVRY Site 3
Any / 28 / .03 / 33 / .04 / 14 / .03 / 14 / .03
Violent / 13 / .02 / 12 / .03 / 6 / .02 / 5 / .02
Non-violent / 19 / .03 / 29 / .05 / 8 / .03 / 12 / .03
Violations / 0 / .00 / 0 / .00 / 1 / .01 / 0 / .00
YLS Site 1
Any / 39 / .04 / 18 / .03*** / 26 / .03 / 13 / .02***
Violent / 10 / .02 / 5 / .02* / 7 / .02 / 4 / .01
Non-violent / 25 / .03 / 13 / .02*** / 22 / .03 / 10 / .02***
Violations / 10 / .02 / 4 / .01 / 1 / .01 / 1 / .01
YLS Site 2
Any / 37 / .06 / 35 / .06 / 31 / .04 / 24 / .05
Violent / 10 / .04 / 8 / .03 / 6 / .02 / 7 / .03
Non-violent / 25 / .05 / 25 / .05 / 25 / .04 / 17 / .04
Violations / 3 / .02 / 3 / .02 / 1 / .01 / 1 / .01
YLS Site 3a
Any / 38 / - / 33 / - / 20 / .08 / 14 / .06
Violent / 14 / - / 8 / - / 6 / - / 4 / -
Non-violent / 31 / - / 30 / - / 15 / .06 / 12 / .06
Violations / - / - / - / - / - / - / - / -

Note. SE = standard error. Rates are marginal means after factoring out covariates except where indicated.

a Standard errors are blank for recidivism categories that did not include covariates in the analyses because chi-squares were used instead. In addition, valid data for probation violations were not available for this site.

*p < .05. **p < .01. *** p < .001.