Forecast Accuracy Scorecard

How our forecasts performed against the Court's decisions. Only forecasts saved before each decision are scored (32 cases), each on the version saved before the decision. Forecast cases were chosen as high-profile cases during OT2025 — 37 of the term's 58 argued cases — not a random sample, so these rates may not generalize to the full docket.

Winner

69%

22/32 cases

95% interval 53–84%

Always pick petitioner

72%

23/32 cases

baseline, same cases

Vote split

34%

11/32 cases

95% interval 19–50%

Justice votes

68%

187/275 votes

95% interval 58–78%

Brier score (lower is better)

0.217

base rate: 0.204

95% interval 0.149–0.293

Bottom line. On 32 genuine forecasts we picked the winner 22 times (69%); always picking the petitioner would have scored 23 (72%). The difference has a 95% interval of -22 to 16 points — with this few cases we cannot show any edge over the base rate yet.

Against baselines

Raw vs. calibrated probabilities

From OT2026 every forecast shows two numbers: the raw probability our method produces, and a calibrated one that pulls it toward the historical petitioner win rate (see what's new for OT2026). The scorecard keeps scoring the raw forecast as published; here both are scored on the same cases.

Calibration

When we said 70%, were we right about 70% of the time? Each dot is a confidence band; dots on the diagonal are well calibrated. Bands with few cases are noisy.

Where we're right and wrong

Why we miss unanimity

Does combining three methods help?

Per-Justice accuracy

Case by case

Pips are each Justice's predicted vote: R right, R wrong, R no published vote, R not scored (took no part, or concurred in part and dissented in part). A concurrence in the judgment counts as a vote for the winning side. ⚑ marks a forecast that contradicted itself; it is scored as published. ⓘ marks a known problem in a case's underlying data, explained on the case page.

Forecast record and integrity

Retroactive back-test (not a forward test)

After the term, we re-ran all 37 OT2025 forecasts “blind” with our final, matured rules: separate model runs that could not see decisions, opinions or our earlier forecasts. They called 31 of 37 winners (84%), against 27 of 37 live.

Treat that 84% as an upper bound, not as forward skill. The test is in-sample: three of the five fixed cases (Chevron, Flowers, Monsanto) came from issue-specific rules written from these very cases. Netting those out, the clean gain is about one or two cases (≈76–78%). Blind runs also reproduced our marquee miss (T.M.) and broke one live hit (Montgomery). The real test is OT2026, forecast live with the frozen-version ledger below.