Blind Collaborative Justice Survey, 2014 [UNITED STATES] (ICPSR 35258)

Version Date: Jun 29, 2017

Cite this study |

Principal Investigator(s):
Carolyn Wong, RAND Corporation; Eyal Aharoni, RAND Corporation

https://doi.org/10.3886/ICPSR35258.v1

Version V1

Slide tabs to view more

Summary

These data are part of NACJD's Fast Track Release and are distributed as they there received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except of the removal of direct identifiers. Users should refer to the accompany readme file for a brief description of the files available with this collections and consult the investigator(s) if further information is needed.

This study conducted an experimental investigation of two potential contributors to invalid testimony within adversarial litigation involving forensic evidence. First, the experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and secondly the lack of input from the relevant scientific community. The study used an experimental survey design with a realistic criminal case to examine the effects of blinding experts to their party representation and consensus feedback from a panel of experts to quantify and reduce testimonial bias.

Citation

Wong, Carolyn, and Aharoni, Eyal. Blind Collaborative Justice Survey, 2014 [UNITED STATES]. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2017-06-29. https://doi.org/10.3886/ICPSR35258.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote

Funding

United States Department of Justice. Office of Justice Programs. National Institute of Justice (2013-IJ-CX-0002)

Subject Terms

Geographic Coverage

Smallest Geographic Unit

None

Restrictions

Access to these data is restricted. Users interested in obtaining these data must complete a Restricted Data Use Agreement, specify the reasons for the request, and obtain IRB approval or notice of exemption for their research.

Distributor(s)

Inter-university Consortium for Political and Social Research
Hide

Time Period(s)

2014-02 (Consensus exercise data), 2014-04-11 -- 2014-04-30 (Justice survey data)

Date of Collection

2014-02 (Consensus exercise data), 2014-04-11 -- 2014-04-30 (Justice survey data)

Data Collection Notes

  1. These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.

Hide

Study Purpose

The purpose of this study was to evaluate the ability of expert blinding and consensus feedback to improve the validity of expert testimony, specifically in the context of forensic science.

Study Design

The study conducted an experiment that consisted of developing a hypothetical criminal case with forensic evidence, carrying out an exercise with a panel of relevant experts to generate a consensus interpretation of the case evidence. The consensus exercise used a questionnaire consisting of the case description and the forensic question. Two rounds of the consensus exercise was needed to reach the unanimous solution, and this solution comprised the feedback for the justice survey.

The justice survey was conducted with a new sample of scientists using the consensus interpretation. The survey was designed and staged on SurveyMonkey, a web-based commercial tool for survey design and administration. Survey participants were randomly assigned to one of three types of party representation (prosecution, defense, or blinded). Approximately half the participants in each condition received consensus feedback in advance of providing a response, and the other half did not. In addition, the group without advanced feedback received an opportunity to change their baseline response following receipt of consensus feedback (a pre/post within-subjects measurement). After the completion of the survey, respondents were invited to a independently hosted website, SelectSurvey.net, to request compensation. This procedure allowed the partition of identifying information from survey responses.

Sample

Consensus exercise: Twelve RAND employees who held doctorates in behavior decision theory, biostatistics, clinical psychology, criminal justice, economics, experimental forensic psychology, industrial and organizational psychology, social psychology, psychology, and statistics.

Justice survey: Fourteen professional scientific societies were targeted. Eligible societies were those with a focus on social/behavioral sciences or statistics and with a primarily United States membership. Nine societies agreed to distribute the survey invitation to their membership list via e-mail or newsletter. 685 accessed the survey and 580 submitted completed surveys.

Time Method

Cross-sectional

Universe

People with an advanced degree in a relevant scientific field where training would normally include exposure to Bayes theorem or equivalent experience.

Unit(s) of Observation

Individual

Data Type(s)

Mode of Data Collection

Description of Variables

Consensus exercise data (JusticeConsensusExerciseDataCollectionInstrument_20140716163600.pdf, n=12) consists of expert's answers to a forensic question.

Justice survey data consists of two Excel files.

N700_RawData_May1_2014 - Archive3.xlsx Excel file contains three worksheets.

  • Values worksheet (n=700, 35 variables) and Labels worksheet (n=700, 33 variables) consists of nearly the same variables with the Labels worksheet's values coded using the coding scheme. These worksheets contain variables on respondent's answer to a forensic question and their confidence in their answers, if a consensus response influenced their final answer, opinion on the survey, how much they would ask for payment for consulting for the prosecution and defense, if their answer would favor the party that hired them, how many times they served as an expert consultant for prosecution and defense, age, gender, race, highest degree earned and field, and other comments on the survey. Only in the Values worksheet are variables on survey exclusion and reason.
  • Summary worksheet contains summary on number of respondents per between-subjects survey condition and a diagram of the conditions.

N580_ProcessedData Oct2014 - Archive.xlsx Excel file (n= 580, 57 variables) contains one worksheet Merged Data. This worksheet contains the same variables as the Labels worksheet in the N700_RawData_May1_2014 - Archive3.xlsx Excel file with the addition of variables on type of survey taken; response to question and confidence level; if respondent has a doctoral degree, has experience, has a forensic degree; and statistics on respondent's answer.

Response Rates

82.9% of people who accessed the survey landing page submitted response data and were thus defined as participants.

Presence of Common Scales

Likert-type scale was used.

Hide

Original Release Date

2017-06-29

Version History

2018-02-15 The citation of this study may have changed due to the new version control system that has been implemented. The previous citation was:
  • Wong, Carolyn, and Eyal Aharoni. Blind Collaborative Justice Survey, 2014 [UNITED STATES]. ICPSR35258-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2017-06-29. http://doi.org/10.3886/ICPSR35258.v1
Hide

Weight

None

Hide

Notes

  • These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.

  • One or more files in this data collection have special restrictions. Restricted data files are not available for direct download from the website; click on the Restricted Data button to learn more.