{
 "note": "Every change to how SCOTUS Analytics makes or scores forecasts, newest first. Each entry says what changed, why, and the evidence. Rendered on /about/ (Methodology); kept in step with METHODOLOGY.md. Forecasts are always made, frozen and scored under the method in force when they were published; a change never rewrites an earlier forecast.",
 "current": "2026.10-OT2026",
 "entries": [
  {
   "version": "2026.10-OT2026",
   "date": "2026-10-06",
   "title": "What's new for OT2026: base-rate anchoring, open-period question counts, a unanimity default, two sentiment readings (Claude Sonnet 5.5 and Jev)",
   "summary": "Changes from the OT2025 post-mortem. OT2025 forecasts did not beat simply picking the petitioner (22 vs 23 of 32), were overconfident, and forecast a unanimous decision only once (wrongly) when nine cases were unanimous. The anchoring, question-count and unanimity changes were back-tested before adoption; anchoring now also accounts for recusals; the research on each Justice is wider; our sentiment classifier moves to Claude Sonnet 5.5; and Jev adds a second sentiment reading that is recorded and scored but not used in the forecast.",
   "changes": [
    {
     "what": "Two probabilities for every forecast: raw and calibrated.",
     "detail": "The raw probability is what our method produces and what the scorecard scores. The calibrated probability pulls it toward the historical petitioner win rate (68%, Supreme Court Database OT2010–24) on the log-odds scale: p = expit(logit 0.68 + 0.4 × (logit p_raw − logit 0.68)). A raw 90% becomes 79%; a call for the respondent stands only if its raw probability is at least 75%.",
     "why": "OT2025 forecasts sat about three times too far from the base rate: calls made at 60–80% confidence came true 59% of the time.",
     "evidence": "Re-fitting the rule without each OT2025 case in turn, it improved our Brier score from 0.217 to 0.213 — but did not beat simply forecasting 68% for every case (0.204). It limits overconfidence; it is not evidence of skill.",
     "links": [
      "/data/forecasts/calibration.json",
      "/accuracy/#raw-vs-calibrated"
     ]
    },
    {
     "what": "With a recusal, the calibration anchors the full Court and then removes the absent Justice.",
     "detail": "When a Justice does not take part, the forecast also estimates the outcome with that Justice voting as their record suggests. That full-bench figure is pulled toward the 68% base rate as usual, and the effect of the absence is then applied on the log-odds scale: logit p = logit anchor(p_full) + (logit p_sitting - logit p_full). The raw probability is still the forecast for the Justices who actually sit.",
     "why": "The base rate averages over absences whose direction is unknown, but a recused Justice's likely vote usually is known. Anchoring only the eight-Justice figure pulls a forecast back toward the petitioner even when the absent Justice was a near-certain petitioner vote. In Suncor v. Boulder County (Alito recused), plain anchoring turned a raw 58% Boulder forecast into a calibrated 58% for Suncor; the recusal-aware figure is 56% Boulder.",
     "evidence": "A reasoned adjustment, not a back-tested one: recusals are too rare for a fit (petitioners won 65.3% of 259 eight-member cases, OT2000-24). It changes only the calibrated figure; the raw probability, which the scorecard scores, is unaffected.",
     "links": [
      "/data/forecasts/calibration.json"
     ]
    },
    {
     "what": "Question counts use the open period of argument only, per lawyer.",
     "detail": "Since OT2021 each lawyer's time ends with a round in which the Chief Justice calls on every Justice in turn. We now leave that round out and divide each side's Justice turns by the number of lawyers who argued for it. Per Justice the count becomes a calibrated probability, used only as a modest shift.",
     "why": "Questions in the round carry no signal about votes, and because every lawyer gets a round, the side with more lawyers (usually the side the Solicitor General supports) drew more questions — and usually won. With the round included, 'the side asked more questions loses' was right in 49.7% of OT2021–25 cases.",
     "evidence": "Open-period counts per lawyer, OT2021–25: winner 62.7% of cases (still below 'the petitioner always wins', 68.5%, so never used to pick a winner); individual votes 64.4%; fitted on OT2010–19, the per-Justice probability adds about 4 points over the base rate out of sample.",
     "links": [
      "/data/arguments/elp-phases.json",
      "/arguments/"
     ]
    },
    {
     "what": "Technical cases default to a unanimous forecast.",
     "detail": "Every case is framed before it is forecast. A procedural or technical case with no federal party is forecast unanimous unless each predicted dissenter has written or joined an opinion on the same question; federal-party technical cases and hybrid cases carry a minimum probability of unanimity. When two splits are equally likely, the more lopsided one is the headline. The publishing step enforces all of this.",
     "why": "Nine OT2025 cases were unanimous and we called none correctly. Most were technical cases forecast at 5-4 to 7-2. The written rule favoring unanimity in technical cases came after those forecasts, and none of them had been framed.",
     "evidence": "On OT2025 the rule would have lifted exact-split accuracy from 11 to 18 of 32 (an optimistic figure: the rule was designed after seeing those cases). In the Supreme Court Database, OT2015–24, non-constitutional cases were unanimous 48–52% of the time, often enough for the default to pay off in 9 of 10 terms.",
     "links": [
      "/data/forecasts/unanimity-rules.json"
     ]
    },
    {
     "what": "Each Justice's pre-Court opinions, confirmation testimony and public statements are part of the per-Justice research.",
     "detail": "Alongside their Supreme Court votes and separate opinions, the research on each Justice now searches the 2,572 lower-court opinions the current Justices wrote before joining the Court, their confirmation-hearing answers and year-end reports (only the nominee's own words count), and their tagged speeches and writings. Evidence is ranked: Supreme Court opinion written > joined > pre-Court opinion written > stated views on method > oral argument.",
     "why": "These sources were already on the site (Justice profiles) but the forecasting research did not use them, so a Justice's own prior writing on a doctrine could be missed when the Supreme Court record was thin.",
     "evidence": "A sourcing change rather than a back-tested rule; its effect will show in the OT2026 per-Justice scores.",
     "links": [
      "/justices/"
     ]
    },
    {
     "what": "Our sentiment classifier now runs on Claude Sonnet 5.5.",
     "detail": "The classifier is unchanged in every other respect: the same seven categories, prompt and inputs (each Justice turn plus which side's lawyer is being questioned) and the same per-Justice fusion formula. Only the model changes, from Claude Sonnet 4.6 to Claude Sonnet 5.5 (claude-sonnet-5-5); every output file records the model. It feeds forecasts made from OT2026 on. Suncor v. Boulder County and Johnson v. Congress, first forecast with Sonnet 4.6, were switched to Sonnet 5.5 on 2026-10-06, before their decisions (the Sonnet 4.6 runs are kept). Neither forecast changed: Suncor stays Boulder, raw 0.58, 4-4; Johnson stays Johnson, raw 0.53, 5-4. In Johnson, Alito's lean toward the government weakened and Kagan's and Barrett's became neutral, but after review sentiment carried no weight in those Justices' probabilities.",
     "why": "On the same OT2025 cases and inputs, Sonnet 5.5 read individual votes better than Sonnet 4.6.",
     "evidence": "14 decided OT2025 cases, 122 votes, same side labels for both models: Sonnet 5.5 88 correct (72%), Sonnet 4.6 78 (64%), 'every Justice votes petitioner' 72 (59%); case-level 95% interval for the gain +4 to +12.5 points. Retroactive (run after the decisions) and mostly divided cases. Jev's rich-context reading got 99 of the same 122, but with more context, so the two are not a like-for-like model comparison.",
     "links": [
      "/data/forecasts/jev-backtest.json",
      "/about/#jev"
     ]
    },
    {
     "what": "An experiment: TypeSafe's Jev reads every OT2026 argument in parallel; it is scored, not used.",
     "detail": "For every OT2026 argued case, each Justice turn of five words or more is sent to Jev (jev-1.13) with the case name, question presented, party names, which party's lawyer is being questioned and that lawyer's previous answer. Jev returns the probability that the Justice doubts that lawyer's position and the probability that the Justice supports it. A Justice's score is the average of doubt minus support, signed so that positive leans petitioner; the lean is its sign. It is published on each case page before the decision, kept out of the forecast, and scored afterwards. The method was fixed on 2026-10-06, before any OT2026 decision. It runs alongside our own classifier (now on Claude Sonnet 5.5); the case page shows both readings side by side. It runs on every argued OT2026 case, whether or not we forecast it, and each case page summarizes Jev's labels: how often each side's lawyer drew doubt or support, the most common label, the most one-sided Justice, the questions it read as most doubtful, and how confident it was. Results are tracked case by case and scored as decisions arrive.",
     "why": "Run after the decisions on 14 OT2025 cases, asking directly whether a Justice doubts or supports the lawyer read individual votes better than our seven-category classifier. Whether that holds on cases not yet decided is the test.",
     "evidence": "OT2025, 14 decided cases, 122 votes: Jev 99 correct (81%) vs our classifier 78 (64%) and 'every Justice votes petitioner' 72 (59%); case-level 95% interval for the gain over our classifier, +10 to +25 points. Optimistic: retroactive, the best of seven variants tried, and 12 of the 14 cases were divided. On the 11 with a pre-decision forecast, our published per-Justice forecasts got 79 of 97 (81%), Jev 81 of 96. Cost about 2 cents per case.",
     "links": [
      "/about/#jev",
      "/data/experiments/jev-ot2026.json",
      "/data/forecasts/jev-backtest.json"
     ]
    },
    {
     "what": "A trained Jev system joins the experiment: a calibrated probability per Justice, recorded before each decision.",
     "detail": "For each argued OT2026 case, Claude Opus 5.5 writes the case's questions (through the API, exactly as in training) from the anonymized opening statements (each side's position, the 2-4 issues with the answer that helps each side, concessions to watch, off-ramps, and the direction the petitioner seeks). Jev then answers about 17 questions on every anonymized exchange (the lawyer's previous answer, the Justice's remark, the reply and the Justice's follow-up), such as doubt, support, friendly question, devil's-advocate testing, own view, pressure for a concession, severity, impatience, limits-testing hypotheticals and the issue lean. Each Justice's answer rates toward each side go into a logistic regression trained on OT2010-2025 votes. It is a third column next to Sonnet 5.5 and the simple Jev reading; it is not used in the forecast. Fixed on 2026-10-07.",
     "why": "The simple Jev reading asked two questions and combined them by hand. Asking more targeted questions about each exchange and learning their weights from past votes gives better-calibrated probabilities.",
     "evidence": "Trained on OT2010-2020, it called 74.1% of OT2021-2024 votes (1,686 votes; ideology baseline 64.1%, question counts 66.4%); refit on OT2010-2024 it called 72.4% of OT2025 votes (427 votes, scored once), log loss 0.541 vs 0.650 for the ideology baseline; its confident calls (probability above 0.75 or below 0.25) were right 88% of the time. Anonymized throughout; content can still identify famous cases, so OT2026 is the clean test.",
     "links": [
      "/about/#jev",
      "/data/experiments/jev-ot2026.json"
     ]
    }
   ]
  },
  {
   "version": "2026.10-parser",
   "date": "2026-10-05",
   "title": "Transcript parser recognizes every Justice since OT2005",
   "summary": "The parser knew only the current nine Justices, so in older transcripts Justices Scalia, Kennedy, Ginsburg and Breyer were dropped and their words folded into the lawyer's turn. Fixed; per-side counts now agree with an independent count of the raw transcripts in 94% of OT2010–19 arguments (was 74%). Current-term forecasts were unaffected.",
   "changes": [],
   "links": [
    "/research/replication/kaufman-2019/"
   ]
  },
  {
   "version": "2026.10-scorecard",
   "date": "2026-10-03",
   "title": "Forecast record and a rigorous scorecard",
   "summary": "Only the version of each forecast saved before the decision is scored, and every version is archived. The scorecard adds baselines (always the petitioner; the historical win rate), Brier score, log loss, calibration, bootstrap intervals and a comparison of the three methods; forecasts saved after a decision are listed and excluded. A diagnostic model of unanimity was added. The finding: on 32 genuine forecasts, no measurable edge over the base rate.",
   "changes": [],
   "links": [
    "/accuracy/",
    "/data/forecasts/scorecard.json",
    "/data/forecasts/ledger.json"
   ]
  },
  {
   "version": "2026.06-gates",
   "date": "2026-06-19",
   "title": "Winner gate recalibrated on scored cases",
   "summary": "After scoring the first blind tests: the sentiment of the Justices' questions is weighted more heavily; the question-pattern composite becomes an input to the winner only at high confidence; and (June 26) the winner is settled from the directly on-point precedent before oral-argument signals are considered, because the side questioned hardest is often the side the Court favors.",
   "changes": [],
   "links": []
  },
  {
   "version": "2026.06-agents",
   "date": "2026-06-12",
   "title": "Multi-agent forecasting pipeline",
   "summary": "Each forecast is built by a sequence of research steps: frame the actual question and assign a track (procedural-technical, ideological or hybrid); map the controlling precedent; research each Justice's record on it; red-team the draft; synthesize the forecast. A guard refuses to forecast a case already decided.",
   "changes": [],
   "links": []
  },
  {
   "version": "2026.06-lessons",
   "date": "2026-06-03",
   "title": "Lessons feedback loop",
   "summary": "Every decided case is scored and its lessons recorded per case and in a cumulative lessons file that later forecasts must consult.",
   "changes": [],
   "links": []
  },
  {
   "version": "2026.03-integrity",
   "date": "2026-03-15",
   "title": "Forecast integrity rules",
   "summary": "Knowledge from after a decision is removed from per-Justice forecasts, and each case page compares the three methods side by side.",
   "changes": [],
   "links": []
  },
  {
   "version": "2026.02-launch",
   "date": "2026-02-08",
   "title": "Three methods",
   "summary": "Forecasts combine the sentiment of each Justice's questions at argument (classified by Claude), Epstein–Landes–Posner question counts, and each Justice's voting record in related precedent from the Supreme Court Database; forecasts are presented as probability estimates.",
   "changes": [],
   "links": []
  }
 ]
}