Katz, Bommarito & Blackman (2017): A general approach for predicting the behavior of the Supreme Court of the United States

Question. Can a single model, using only information available before each decision, predict the Court's decisions and every Justice's vote across its whole history, term after term, as Justices come and go?

Design. A simulated forecast of every term from 1816 to 2015. Before each term the model is retrained on all earlier terms only, then predicts every vote and case in that term. It is scored against a strong baseline: the most frequent outcome of the previous ten terms, a window the authors picked because it was close to the best possible in hindsight.

Data

• Every Justice vote in the Supreme Court Database's Legacy release, 1791–2015: about 244,000 votes in 28,000 cases, of every decision type.
• Each vote is coded affirm, reverse or other from the vote and the Court's disposition (a dissent from a reversal counts as affirm; jurisdictional dissents and unclear dispositions are other). A case counts as reversed when the Court reverses or vacates.
• About 1,500 inputs per vote: the Justice, natural court, argument month, the parties (raw and grouped), jurisdiction, agency action, source and origin court mapped to circuit, lower-court disagreement and disposition, cert reason, issue, issue area, law type and law, plus the decision month and the weeks from argument to decision.
• "Behavior" inputs computed from earlier terms only: each Justice's and the whole Court's reversal rate, ideological direction and agreement with the majority, in the previous term and over all prior terms, the differences between Justice and Court, and how far the lower court's ruling sits from the Justice's usual direction.

Method

1. A random forest: hundreds of decision trees, each grown on a random sample of votes and inputs, voting together.
2. Retrained at the start of every term on all earlier terms, with 125 trees plus 5 for each term elapsed. A 3-fold grid search picks the minimum leaf size (1 or 2).
3. Each vote is predicted as affirm, reverse or other. The case is predicted reversed when at least half the predicted votes are to reverse.
4. Baseline (M = 10): predict the most frequent vote outcome of the previous ten terms.

What they found

The takeaway. A general-purpose model can follow the Court across changes of personnel and centuries of history, and beat a well-tuned baseline over the long run; the edge is a few points, and it was shrinking in the most recent terms the paper covers.

• Daniel Martin Katz, Michael J. Bommarito II & Josh Blackman, A general approach for predicting the behavior of the Supreme Court of the United States, PLOS ONE 12(4): e0174698 (2017). DOI →
• Replication code and per-case predictions: github.com/mjbommar/scotus-predict-v2. GitHub →
• An earlier preprint (arXiv:1407.6333, 2014) used extremely randomized trees on 1953–2013; the PLOS ONE paper supersedes it. arXiv →

The authors published their code and every prediction, which makes an exact rebuild possible. Reading it settles three details the paper leaves open:

• Two inputs come from the decision itself. The month the decision came down and the weeks between argument and decision are both inputs, and in the authors' final model the delay is among the three most important. The paper explains that the delay can be updated week by week while a case is pending, but the published backtest uses the final value. We report a version without both.
• The forest is rebuilt every term. The paper describes a forest that grows by five trees a term. In the code, the grid search fits a fresh copy each term, so each term gets a newly grown forest of 125 + 5 × terms trees. We do the same.
• The common outcome is weighted up. Class weights are set to each outcome's share of the training votes, which tilts the forest toward the most common vote.

• The paper reproduces. With the authors' code as a guide, our rebuild matches their published predictions within a point and makes the same call on 87.6% of cases.
• The edge has shrunk. Against the ten-term baseline the model is +5.1 points over OT1971–OT2004, +1.5 points over OT2005–OT2014 and +2.0 points over OT2015–OT2024. As the Court came to reverse more often, the baseline got harder to beat.
• On today's Court it is nearly an "always reverse" rule. It predicted reversal in 227 of 233 argued cases in OT2022–OT2025 and in all 57 in OT2025, so in that term it scores exactly what always predicting reversal scores. Its few other calls give it a +2.6 points edge over the four terms, too small to tell from luck.
• Decision timing barely matters for cases. Dropping the decision month and the argument-to-decision delay changes the call in only 10 of 1,528 cases (OT2005–OT2025), with the same accuracy (70.7%). It helps slightly with individual votes: 65.0% with them, 64.7% without.
• What drives it. In the OT2025 model the most important inputs are how far the lower court's ruling sits from each Justice's usual ideological direction, then the argument-to-decision delay and the Justices' past voting patterns:

1. Lower court's direction minus the Justice's career voting direction 3.8%
2. Lower court's direction minus the Justice's voting direction last term 3.6%
3. Weeks from argument to decision 3.0%
4. Justice's voting direction last term 1.8%
5. Court's direction minus the Justice's, last term 1.8%
6. Justice's career voting direction 1.7%
7. Court's direction minus the Justice's, career 1.7%
8. Justice's reversal rate last term 1.6%