O.J. says "duh": New study shows juries often get it wrong

A Northwestern University statistics professor, Bruce Spencer, studied 271 criminal cases and found "that juries gave wrong verdicts in at least one out of eight cases."

To conduct the study, Spencer employed a replication analysis of jury verdicts, comparing decisions of actual jurors with decisions of judges who were hearing the cases they were deciding. In other words, as a jury was deliberating about a particular verdict, its judge filled out a questionnaire giving what he or she believed to be the correct verdict.

“Consider the analogy to sample surveys, where sampling error is estimated even though the true value may never be known,” Spencer said. “The key is replication. To assess the accuracy of jury verdicts, we need a second opinion of what the verdict should be.”

By comparing agreement rates of judges and juries over time and across jurisdictions, and even across types of cases, Spencer’s statistical analysis could give insights into the comparative accuracy of verdicts in different sets of cases.

A draft of his paper, to be published in the Journal of Empirical Legal Studies, can be found here (PDF).