Jurimetrics

'Judging the Judiciary by the Numbers: Empirical Research on Judges' by  Jeffrey J. Rachlinski and Andrew J. Wistrich in (2017) 13 Annual Review of Law and Social Science asks

Do judges make decisions that are truly impartial? A wide range of experimental and field studies reveal that several extra-legal factors influence judicial decision making. Demographic characteristics of judges and litigants affect judges’ decisions. Judges also rely heavily on intuitive reasoning in deciding cases, making them vulnerable to the use of mental shortcuts that can lead to mistakes. Furthermore, judges sometimes rely on facts outside the record and rule more favorably towards litigants who are more sympathetic or with whom they share demographic characteristics. On the whole, judges are excellent decision makers, and sometimes resist common errors of judgment that influence ordinary adults. The weight of the evidence, however, suggests that judges are vulnerable to systematic deviations from the ideal of judicial impartiality.

The authors comment

Judges are the axle on which the wheels of justice turn. They manage pretrial proceedings, mediate settlement conferences, rule on motions, conduct bench trials, supervise jury trials, take guilty pleas, impose criminal sentences, and resolve appeals. In the process, they find facts, make or apply law, and exercise discretion. Judges wield enormous power and society therefore rightly expects much of them. Judges must be fair minded, impartial, patient, wise, efficient, and intelligent (Wistrich, 2010). They must set aside their politics and their prejudices, make rational decisions, and follow the law. (See, e.g., American Bar Association, Model Code of Judicial Conduct, 2011, Rules 1.1, 1.2, 2.2, 2.3, 2.4, 2.5, 2.8). But is it possible for judges to perform as we expect?

The answer to this question remains somewhat uncertain. Twenty years ago, Lawrence Baum (1997, p. 149) concluded, “Despite all the progress that scholars have made, progress that is accelerating today, we are a long way from achieving truly satisfying explanations of judicial behavior.” Much more research has been conducted since then, but judicial behavior still remains something of a mystery. Some scholars argue that judges behave rationally but make decisions that further their self-interest ( Epstein et al. 2013). That assertion, however, raises as many questions as it answers: What do judges see as their self-interest? Are fairness and impartiality their primary goals? What incentives do judges really face? After all, they rarely lose their positions and seldom get promoted. And even if judges primarily strive for fairness and impartiality, do they achieve these goals?

Research on human judgment and choice indicates that most people face cognitive limitations that lead them to make choices that do not consistently further their own ends (Ariely 2009). People commonly rely on intuition and simple shortcuts (or  heuristics) to make choices (Kahneman 2011). Heuristics can be effective and surprisingly accurate (Gigerenzer and Todd 1999), but can also lead to predictable mistakes when over-applied or misused. These problems plague professionals as well. Research on doctors, dentists, accountants, futures traders, and others shows that they all fail to live up to an idealized standard of judgment in many settings ( Ariely 2009). It would be surprising if judges are any different.

The available research on judges suggests that they sometimes f all short of the lofty ideal to which society holds them. A growing body of research supports the conclusion that although judges are often excellent decision makers, they have vulnerabilities. At the outset, we know that in some areas of law, judicial decisions are too chaotic. A study of immigration asylum decisions, for example, reveals that some judges grant asylum in a high percentage of cases while others almost never grant asylum (Ramji-Nogales et al. 2007). Asylum outcomes thus turn on the random assignment of a case to one judge or another. Decisions concerning whether to grant leave to appeal or to allow release on bond in immigration cases are similarly erratic ( Rehaag 2012; Ryo, 2016). Concerns about variation in conviction rates have also long haunted criminal law (Weisselberg and Dunworth, 1993). Even in criminal sentencing decisions in federal court, in which a highly structured set of guidelines cons trains judges, variation remains robust ( Scott 2011). Judges do not seem to decide as reliably as might be hoped or expected. Worse still, the variation does not just arise from chaos or a lack of meaningful standards, it arises from systematic vulnerabilities in how judges think.

This article surveys the empirical research that assesses whether judges live up to the standards of their profession. The evidence accumulated to date reveals that judges fall short in predictable ways. First, as the legal realists feared, judges’ personal characteristics influence their decision making. Specifically, the research indicates that when cases raise issues that are salient to judges’ personal characteristics, they do not consistently put their characteristics aside. Second, judges overreact to mechanisms of accountability, such as appellate review, retention, and promotion. Third, judges rely too heavily on intuitive ways of thinking that can be misleading. Fourth, in making decisions, judges sometimes rely on factors outside the record, including inadmissible evidence, their emotional reactions, and prejudices.

To be fair to judges, they labor under a great deal of academic scrutiny. The existing research on judicial decision making probably focuses too heavily on judicial failings. Scholars conduct their research with an eye towards showing that judges are politically motivated or biased. This is understandable, given the ideal of neutral judging that society expects from judges, but the emphasis on deviations likely makes judges seem worse than they are. The research includes several studies in which judges adhere to an ideal norm of neutrality, and we certainly include these in our review. No studies really provide usable estimates of how many cases are skewed by politics, prejudice, or other misjudgment, and the research does not support a means of making a reasonable estimate. The circumstances under which judges deviate from the norm are nevertheless worth exploring, not to make judges look bad, but to identify potential ways they might improve.

In reaching our conclusions, we review a diverse array of both experimental and field studies of judicial decision making. We set aside judges’ autobiographies and biographies, interviews of judges, careful parsing of individual opinions, and judges’ own accounts of how they make decisions. Such undertakings can provide valuable insights, but our focus lies on systematic empirical accounts of judicial decision making. These include archival studies of actual decisions and experiments or simulations using hypothetical cases. Although most research on judges emphasizes decisions of the US Supreme Court (especially since the Second World War), our focus lies with the state courts, lower federal courts, and a handful of international studies. Although the US Supreme Court is important, of course, it resolves few cases and represents only a tiny window into the judicial decision-making process. Each of the studies we incorporate into our analysis involves vastly more judges than the 39 people who have served on the Supreme Court in the last 70 years. The focus on the Supreme Court also tends to emphasize the role of politics in judging. Political influence is only one way judges can fail to meet the demands of their roles. We discuss this concern but expand upon it.

In New Zealand The Wheels of Justice: Understanding the Pace of Civil High Court Case by Bridgette Toy-Cronin, Bridget Irvine, Kayla Stewart and Mark Henaghan comments 

Delays in the court process are a key obstacle in accessing justice. Delay creates costs; not only in the loss of time but also financial and psychological costs. These costs are borne by the litigants, the economy, and the public purse. This is the first major New Zealand study to investigate the pace of High Court civil cases and to examine if, and where, delays might occur. 

In this report, we look at both the overall length of cases, and we focus on various points in the life of a case where delay might occur. We have used mixed methods to study these issues: a quantitative analysis of data provided by the Ministry of Justice, an analysis of physical court files, and interviews with lawyers, judges, court staff, and litigants. 

Determining the overall length of a case is a more complex task than it appears on its face, particularly as there are limitations to the data recorded by the Ministry of Justice. Where possible, we have used our analysis of the physical court files to overcome these limitations and evaluate case length. On average, a case filed in the High Court will conclude within 191.5 days. General proceedings, one of the types of civil proceedings heard by the High Court, frequently exceeded the average case length, taking an average of 381 days to conclude. As general proceedings were the longest class of cases and account for 29 per cent of the High Court’s total caseload, the report focuses on this case type. Study participants agreed that most general proceedings should not exceed two years; only 18 per cent of general proceedings exceeded this limit. 

Analysing case length alone, however, cannot answer all questions about delay. Delay can occur in extremely short cases; conversely, for some very long cases the passage of time could not be conceived as delay. In fact, we précised several long cases that had no evidence of delay. These included cases that were ‘parked’ for various reasons: waiting for a related case to be resolved, an appeal to be heard, remedial work to be undertaken, or a settlement negotiated. Some cases just needed more time to be ready for trial, especially cases involving multiple parties, or with complex evidentiary issues. While lengthy, these cases were not necessarily delayed. 

Other cases – long and short – exhibited evidence of delay. Interviews with the participants helped to tease out the nature of this delay. The lack of judicial time to promptly hear fixtures (interlocutory and substantive) and deliver judgments was of particular concern. The unavailability of litigation participants, especially experts, also slows the pace of a case. Errors by registry were also evident; while rare, these errors can delay case progression. Finally, litigation involves a range of participants: litigants, lawyers, witnesses, court staff, and judges. The behaviours of any of these participants in the process can affect pace. For example, litigants, whether represented or unrepresented, can create delay for strategic reasons; lawyers preparing court documents late or to a poor standard can create delay. We canvas the interplay between these litigation participants and consider how these relationships can affect pace. 

When considering solutions to the causes of delay the fundamental purposes of the court must be kept to the fore: to secure just outcomes between parties, publicly state the law, reinforce norms, and limit executive power. The court is a complex organisation. There are many participants who each respond to their own pressures and incentives. Any solutions must take into account this complexity. Proposed reforms should be carefully considered and approached cautiously. 

Before firm recommendations can be made, further analysis of this data is required. A number of possibilities, however, have emerged at this preliminary stage. Many of these reforms centre on the case management process, including: earlier identification of issues in dispute, greater inclusion of litigants earlier in the process, improving the timing and methods of eliciting witness evidence, considering judicial specialisation, and setting firm timetables. Another key area for further research is initiatives to lower or better plan the cost of legal representation, which has a close but complex relationship with the pace of litigation. Other possible reforms focus on the court’s broader operations, including: protecting judgment writing time, and maximising the advantages that can be harnessed from modern technology. There is an urgent need to improve data about who uses our courts, whether or not they are represented, and how their cases proceed. Without this information, we are unable to design a civil justice system that responds to the needs of those using the court and that protects its important public function.