We built real win-probability models for two sports from public box-score data, tested them strictly out-of-sample, and held each to the only bar that counts: the market's price. The two sports answer in opposite shapes — and neither model beats the market.
| ▲Skill — passing / QB rating | AUC 0.658 |
| Line — rush eff, sacks, TFL/QB-hits | AUC 0.618 |
Out-of-sample AUC of each pre-game rating (0.50 = coin flip). Skill clearly leads.
A logistic win-probability model on those ratings, tested out-of-sample: Brier 0.221 vs a 0.249 baseline — 11.4% skill, 64.0% accuracy over 447 games. Genuinely predictive.
Against Polymarket's moneyline, out-of-sample over 443 games: model Brier 0.220 (64.1%) vs market 0.200 (69.8%). The market wins — it prices in injuries, lines and sharp money the model never sees.
| ▲Pitching — starter ERA/WHIP + staff | AUC 0.581 |
| Offense — rolling runs scored | AUC 0.532 |
Out-of-sample AUC of each pre-game rating (0.50 = coin flip). Pitching clearly leads.
A logistic win-probability model on those ratings, tested out-of-sample: Brier 0.246 vs a 0.250 baseline — 1.6% skill, 55.7% accuracy over 970 games. Barely better than a coin flip — the sport is near-random game to game.
Against Polymarket's moneyline, out-of-sample over 640 games: model Brier 0.246 (56.1%) vs market 0.247 (55.8%). A statistical tie (paired t = -0.2) — and pre-vig, so a real bet still loses the spread. No edge.
In the NFL, passing quality — not the line of scrimmage — drives winning, and a public model has real skill (~11.4%). But the market still clearly beats it: there's signal to capture, and the market captures more.
In MLB, pitching and offense matter about equally and both barely beat a coin flip. The model ties the market — not because it's good, but because baseball is nearly random game to game and there's little for anyone to know. The sport itself is the ceiling.
Different reasons, same bottom line — a public-data model doesn't out-price these markets — reached from a completely fresh direction, exactly like the stock funnel and the arena.
Every model is opened up on its own page — the data it eats, the machinery step by step, its live weights where they exist, one fully worked prediction, and exactly which file each knob lives in. Written so someone who has never seen a model can follow, and so tweaking one starts from understanding instead of archaeology.
Not a backtest — this logs each upcoming game's model probability and the market's price before first pitch, then scores it when the game ends. A record hindsight can't fake, accruing through the season.
Over 32 scored games so far: model Brier 0.248 (50%) vs market-at-prediction 0.252 (53%). Dead even so far — as the backtest predicted.
Against the closing line — the harder bar, captured in the last ~90 minutes before first pitch — over 48 games: model Brier 0.254 (46%) vs close 0.257 (48%).
| date | matchup (starters) | model | market | close | result |
|---|---|---|---|---|---|
| 08-01 | MIN @ SEAZebby Matthews vs Bryan Woo | 55% | — | — | — |
| 08-01 | BOS @ LADRanger Suarez vs Yoshinobu Yamamoto | 59% | — | — | — |
| 08-01 | SF @ SDCarson Whisenhunt vs JP Sears | 56% | — | — | — |
| 08-01 | DET @ ATHCasey Mize vs Jeffrey Springs | 43% | — | — | — |
| 08-01 | MIL @ LAARobert Gasser vs Ryan Johnson | 42% | — | — | — |
| 08-01 | KC @ COLMichael Wacha vs Tomoyuki Sugano | 52% | — | — | — |
| 08-01 | TEX @ HOUJacob deGrom vs Hunter Brown | 51% | — | — | — |
| 07-31 | WSH @ ATLMiles Mikolas vs Bryce Elder | 59% | 66% | — | voided |
| 07-31 | MIA @ NYMJanson Junk vs Freddy Peralta | 50% | 54% | — | — |
| 07-31 | CHW @ TBErick Fedde vs Nick Martinez | 55% | 55% | — | — |
| 07-31 | ARI @ CLEMitch Bratt vs Tanner Bibee | 58% | 54% | — | — |
| 07-31 | STL @ TORKyle Leahy vs Dylan Cease | 53% | 56% | — | — |
| 07-31 | PIT @ CINBraxton Ashcraft vs Hunter Greene | 46% | 53% | — | voided |
| 07-31 | SEA @ LADBryce Miller vs Roki Sasaki | 56% | — | — | — |
| 07-31 | SF @ SDRobbie Ray vs German Marquez | 52% | — | — | — |
| 07-30 | PIT @ CINPaul Skenes vs Hunter Greene | 45% | — | — | voided |
| 07-30 | MIA @ NYMEury Perez vs Nolan McLean | 51% | — | — | — |
| 07-30 | WSH @ ATLFoster Griffin vs Bryce Elder | 53% | — | — | voided |
| 07-30 | CHC @ STLShota Imanaga vs Andre Pallante | 48% | — | — | — |
| 07-30 | NYY @ CHWRyan Weathers vs Sean Burke | 54% | 51% | — | — |
| 07-30 | KC @ MINNoah Cameron vs Bailey Ober | 59% | 56% | — | — |
| 07-30 | TEX @ TBNathan Eovaldi vs Shane McClanahan | 55% | 55% | — | — |
| 07-30 | SEA @ LADEmerson Hancock vs Justin Wrobleski | 60% | — | — | voided |
| 07-30 | BOS @ ATHPatrick Sandoval vs Jacob Lopez | 49% | — | — | BOS won |
Home-team win probability. market is the price when the prediction was logged — often days early, often not yet listed; close is the price captured in the last ~90 minutes before first pitch, the harder bar. Rows are never edited or deleted — a game that never resolves as predicted (postponed, canceled), or a row whose recorded market price can't be trusted, is voided: kept in the table, counted toward nothing.
Correction, 2026-07-21: the first days of ledger rows were logged by a market matcher that could pair a game with a different game's price — MLB series share a team pair, and the matcher keyed on the pair alone (it also read "Red Sox" as the White Sox). Every pre-fix row that carried a market price is voided above rather than silently corrected; the model probabilities were unaffected. The market comparison restarts from rows logged after the fix, which matches by scheduled start time.
