Instilling a Culture of Continuous Learning From Criminal Justice System Errors: A Multi-Stakeholder Sentinel Event Review of Process in Philadelphia, Pennsylvania, 2006-2016 (ICPSR 38188)

Version Date: Jul 28, 2025 View help for published

Principal Investigator(s): View help for Principal Investigator(s)
John F. Hollway, University of Pennsylvania. Quattrone Center for the Fair Administration of Justice

https://doi.org/10.3886/ICPSR38188.v1

Version V1

Slide tabs to view more

In January 2016, the Quattrone Center for the Fair Administration of Justice at the University of Pennsylvania Law School was awarded grant 2015-R2-CX-K040 for a proposal entitled "Instilling a Culture of Continuous Learning from Criminal Justice System Errors: A Multi-Stakeholder Sentinel Events Review of Process in Philadelphia." The work performed under the grant built upon a successful pilot program supported by NIJ, in which a group of criminal justice professionals in Philadelphia, PA conducted a multi-stakeholder sentinel event review (SER) of a criminal case in the United States. The Quattrone Center proposed to build upon the pilot project by implementing the Philadelphia Event Review Team (PERT) as a more permanent working group to identify cases of error in the Philadelphia criminal justice system and conduct sentinel event reviews of those cases over the three-year period of performance for the grant.

The PERT formed a group of dedicated criminal justice professionals across multiple agencies - the Philadelphia District Attorney's Office, the Philadelphia Police Department, the Defender Association of Philadelphia, the 1st Circuit Court of Common Pleas of Pennsylvania, and others - who agreed to meet regularly and review cases where all of the participants felt an undesired outcome had occurred.

The process for conducting the second SER was lengthy and circuitous, and was slowed by a number of factors: the churn of elected and appointed personnel in the participating agencies, the overhang of potential impending litigation on cases being reviewed for possible selection, and in one instance, the inability to waive attorney/client privilege for a defendant whose treatment by the system was viewed as suboptimal by the PERT. Both the successes and the challenges of SERs in criminal justice as experienced by the PERT are set forth below.

The SERs conducted as part of this project followed the definition of SER set forth in the NIJ publication Mending Justice: Sentinel Event Reviews. Sentinel events, which have a history of successful application in fields as diverse as aviation, healthcare, and military operations, are multi-stakeholder reviews of instances of error in complex human systems that seek to reveal and understand the root cause(s) of such "never events" as plane crashes, surgical errors, and other accidents. They involve a formal procedure to review errors in a non-blaming atmosphere and determine ways to avoid such errors in the future.

Applying the methodology of Sentinel Event Reviews (SER) to the criminal justice system involves reviewing a criminal investigation and prosecution that demonstrated some "error," as defined by the participants of the SER. The PERT took the position that any wrongful conviction, cold case, officer-involved shooting, or death of an individual in the custody of the state was by definition a case of error; this definition is completely separate from an assessment of blame and simply held that any instance of these outcomes was per se unwanted, even if such an outcome may have resulted by good faith professionals acting within established training and protocols. The PERT attempted to conduct SERs of selected cases in the criminal justice system using principles of "just culture review," defined as "a culture that recognizes that competent professionals make mistakes and acknowledges that even competent professionals will develop unhealthy norms (shortcuts, "routine rule violations"), but has zero tolerance for reckless behavior." (Agency for Healthcare Research and Quality Glossary)

A substantial question regarding the application of SERs to criminal justice was whether the adversarial nature of the criminal justice system, which is different than the structure of the systems in which SERs were originally developed, would be a barrier to their successful application in criminal justice. The adversarial culture unique to the criminal justice process creates additional challenges, including the complexity of gaining consensus from cross-agency participants on the definition or occurrence of errors, the ability to define shared goals within a system, and the ability to implement reforms that require changes in agencies other than one's own. Within that context, the work performed under this grant provides additional information to help answer the questions posed by, and the methodology outlined in, Mending Justice--namely, (1) how might SERs be applied to the criminal justice system; (2) can they effectively reduce future errors; and (3) is this approach sustainable?

The datasets are the result of two data requests. The first data request asked for appeals court cases for the Philadelphia County Court of Common Pleas involving criminal cases with an appeal filed from 1/1/2011 through 12/31/2015 as recorded in the Pennsylvania Appellate Court Management System (PACMS). The second data request asked for data on case level data related to the criminal cases disposed in the Philadelphia County Court of Common Pleas from 11/17/2006 through 11/17/2016 with one of the following final case dispositions: dismissed, judgement of acquittal, mistrial, nolle prossed, nolle prossed (case dismissed), quashed, demurrer sustained, or withdrawn. For each kind of data (cases, case dispositions, offenses, offense dispositions, docket entries) there are two files, one for each of the data requests. The two files for each type of data share the same structure and fields.

Hollway, John F. Instilling a Culture of Continuous Learning From Criminal Justice System Errors: A Multi-Stakeholder Sentinel Event Review of Process in Philadelphia, Pennsylvania, 2006-2016. [distributor], 2025-07-28. https://doi.org/10.3886/ICPSR38188.v1

Export Citation:

  • RIS (generic format for RefWorks, EndNote, etc.)
  • EndNote
United States Department of Justice. Office of Justice Programs. National Institute of Justice (2015-R2-CX-K040)

Zip Code

Hide

2006-11-17 -- 2016-11-17
2016-12-25
Hide

This project evaluated the ability of a multi-stakeholder, cross-agency review team, the Philadelphia Event Review Team (PERT), to regularly prioritize and evaluate both Sentinel Events and more routinized errors in the administration of criminal justice.

To facilitate the identification of cases of error, the Quattrone Center also worked with the Administrative Office of the Pennsylvania Courts (AOPC) to create the Case Review Database, or CARD, a unique data set of cases believed to have unintended outcomes that occurred in the regular administration of criminal justice, and which was designed to allow novel data analytics of errors and potential preventative steps. The PERT and AOPC designed a database of approximately 5000 criminal cases adjudicated by Philadelphia courts and having procedural outcomes that reflected potential undesired outcomes (e.g., cases that were dismissed by the District Attorney's Office after a jury was empaneled; convictions that were overturned on all charges and for which no subsequent criminal charges were pursued, etc.).

The members of the PERT were assembled by the Quattrone Center at Penn Law with assistance from the Philadelphia County Criminal Justice Advisory Board, a sitting organization that consists of a broad cross-section of criminal justice stakeholders in Philadelphia that was supportive of the goals of the PERT. Regular stakeholders included: Philadelphia Police Department (PPD); Philadelphia District Attorney's Office (DAO); Defender Association of Philadelphia; and First District Court of Common Pleas of Philadelphia (CCP). Representatives from the Quattrone Center briefed the CJAB on multiple occasions about the progress of the PERT, and sought out the CJAB for assistance on adding new members for specific cases that were being considered for review (e.g., the CJAB helped the Quattrone Center connect with the Philadelphia Department of Behavioral Health when a case dealing with defendants with behavioral health challenges was identified as potentially ripe for SER). Members also included the Criminal Justice Coordinator for the City of Philadelphia and prominent advisors from the criminal defense bar who were not affiliated with the Defender Association.

The PERT proceeded with the discovery of cases to review in three ways. First, member agencies were asked to nominate cases that had affected their personnel and were believed to be cases of error and/or cases that had resulted in undesirable outcomes. Second, the Quattrone Center conducted a review of Philadelphia cases in the National Registry of Exonerations and a historical review of media reports on overturned Philadelphia criminal cases. Third, the Quattrone Center and the AOPC generated the CARD and the Quattrone Center began to conduct searches of the CARD to reflect areas of interest to the PERT participants.

Criminal court cases from the Philadelphia County Court of Common Pleas.

Court Cases

Unified Judicial System of Philadelphia

Variables in this study include information about appeals, case and offense dispositions, and docket entries from the Philadelphia County Court of Common Pleas.

Hide

2025-07-28

2025-07-28 ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection:

  • Checked for undocumented or out-of-range codes.

Hide

Notes

  • The public-use data files in this collection are available for access by the general public. Access does not require affiliation with an ICPSR member institution.