The outcomes evaluations were designed to examine whether implementation of each Safe Start intervention was associated with individual-level changes in specific outcome domains. The evaluation utilized an intent-to-treat approach that was designed to inform policymakers about the types of outcomes that could be expected if a similar intervention were to be implemented in a similar setting. To prepare for the evaluation, the sites worked together with the national evaluation team to complete a "Green Light" process that developed the specific plans for the intervention and assured that the evaluation plan would align well with the intervention being offered and would be ethical and feasible to implement.
As a result of the Green Light process, a rigorous, controlled evaluation design was developed at each site, either with a randomized control group (wait list or alternative intervention) or a comparison group selected based on similar characteristics. Three sites had more than one study being conducted in their setting, as they delivered two different interventions to different groups of individuals. Overall, there were 18 separate evaluations of interventions within the 15 sites. Most sites utilized an experimental, randomized design (13 of 18), with RAND standardizing and monitoring the randomization procedures. Pre-intervention baseline data were collected on standardized, age-appropriate measures for all families enrolled in the studies. Longitudinal data on families were collected for within-site analysis of the impact of these programs on child outcomes at 6, 12, 18, and 24 months post-enrollment.
The 15 sites collected data with initial training and ongoing support from RAND. Specifically, in order to standardize procedures across each of the 15 Safe Start sites, the RAND evaluation team developed detailed data collection procedures and forms. The supervisor and interviewer training manuals described each step of the data collection process. Using these manuals, the RAND team provided initial on-site data collection trainings for supervisors and research staff employed by each of the 15 Safe Start sites. The sites then implemented the data collection procedures and trained new data collection staff (when turnover occurred). The RAND team provided oversight and delivered refresher training sessions by conference call or on site, as needed. The sites mailed data on a monthly basis to RAND for data entry, cleaning, and analysis.
Data sources for the outcome evaluation were primary caregiver interviews, child interviews (for ages 3 and over), and family/child-level service utilization data provided by the Safe Start program staff. Measures for caregivers and children (ages 3 and up) were assembled into two batteries: a caregiver assessment battery and a child assessment battery. Caregivers completed a battery of instruments that comprised between 95 and 249 items, depending on the age of the child. All caregiver assessments were interviewer administered. Child assessments were interviewer administered for ages 3 through 10. Children ages 11 and older completed a self-administered assessment packet, but research staff was available to assist the child as needed. The child assessment is comprised of 36 to 165 items, depending on the age of the child. The child assessment battery also varied by the age of the child. A Family Status Sheet (FSS), which documented services received and current status of the family's engagement with services, was also completed for all families at each assessment point.
The Office of Juvenile Justice and Delinquency Prevention selected 15 program sites across the country to implement a range of interventions for helping children and families cope with the effects of children's exposure to violence. Program sites were located in the following 15 communities:
Each site recruited participants into experimental or quasi-experimental evaluation studies. Criteria for admission into the studies varied by site. Generally, convenience samples were taken of those who met eligibility criteria at each site.
Across all 15 sites, the dataset is comprised of 5,951 cases including:
ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:
Although the 18 separate evaluations of interventions within the 15 sites all focused on children exposed to violence, they varied considerably in terms of the type of intervention, the setting in which it was offered, and the group of children and families targeted for the intervention. For more in-depth information on the Safe Start Promising Approaches (SSPA) outcomes evaluation including program-specific details, users should refer to the
To assess outcomes at each site, the research team used a set of measures that captured background and contextual factors, as well as a broad array of outcomes, including PTSD symptoms, depressive symptoms, behavior/conduct problems, social-emotional competence, caregiver-child relationship, school readiness/performance, and violence exposure.
Three measures were completed by caregivers to capture background and context.
The research team used two measures to assess child PTSD symptoms, one reported by caregivers for young children, and the second reported by children themselves.
Depressive symptoms in children age 8 and older were collected using one self-report instrument -- the Children's Depression Inventory (CDI; Kovacs, 1981).
To assess internalizing and externalizing behavior problems and delinquency, the research team used several measures and combined them using advanced psychometric techniques to develop a score that could be used across a broader age range.
Measures of affective strengths, school functioning, cooperation, assertion, self-control, and social and emotional competence in general were selected from four scales, two of which have different versions for different age ranges and respondents.
Measures of parenting stress and family involvement were used to assess caregiver-child relationship.
To assess the general domain of school readiness and performance, the Woodcock-Johnson III scale (WJ-III; Blackwell, 2001) was used. The research team used this measure for children between the ages of 3 and 18 and chose three tests: Letter-Word Identification, Passage Comprehension, and Applied Problems.
Three measures were used to capture violence exposure in children and caregivers.
Terms of use are available at http://www.icpsr.umich.edu/icpsrweb/ICPSR/studies/34740/terms
AVAILABLE. This study is freely available to the general public.