Four backtest "edges" that died live

We've been running a statistical model across more than 50 football leagues since February 2026, posting every bet before kick-off and settling it in public. The profit is the least interesting part of that. What's worth writing down is how many edges looked bulletproof in backtest and then quietly stopped working the moment they met a real market. These are the four that fooled us hardest.

1. Look-ahead bias hides in the pipeline, not the logic

Our best backtested strategies were built on snapshot features, meaning data captured at a particular point in a match or a trading day. The modelling logic was fine. The pipeline was the problem. Some of those snapshots carried information that wouldn't have existed yet at the moment we'd have placed the bet, in one case an odds tick that arrived slightly later.

We rebuilt the simulation so it could only see what was actually knowable at the point of decision. Two strategies that had looked strong came out roughly breakeven. We hadn't touched the model. All we'd changed was what the clock was allowed to know.

2. Your biggest edges are usually your model being wrong

We'd assumed bet quality scaled with disagreement, so the wider the gap between our price and the market's, the better the bet. Live results said otherwise. When we bucketed outcomes by edge size, bets showing 30 to 40% edge and above did materially worse than the ones sitting in the 5 to 15% band.

That makes sense in hindsight. Extreme disagreement usually means something has gone wrong on our side, such as a stale odds reading, a badly estimated probability, or a market too thin to trust, rather than the bookmaker having made a large mistake. We now treat an oversized edge as a warning rather than a green light.

3. Winner's curse against a sharp market is brutal

Bet into an efficient market and the situations where the price looks wrong in your favour are, disproportionately, the situations where the market knows something you don't. Late team news. A lineup leak. Money from someone better informed than you. Measured against a sharp reference line, our flagged bets showed textbook adverse selection.

The fix wasn't a better model. It was an agreement gate: we only fire when a soft book is out of line and the sharp reference broadly agrees with our own estimate. That one filter recovered more expected value than any amount of retraining did.

4. A probability floor beats an edge filter

Filtering on expected value alone quietly selects for the model's own overconfidence. The fattest EV numbers come from the priciest longshots, which is exactly where a probability estimate is least reliable and where a small calibration error costs the most.

So we made probability the primary signal with a hard floor under it, and demoted edge to a secondary condition. That change did more for live results than retraining ever did. With money on the line, a well calibrated probability is worth more than a marginally more accurate one.

What the four have in common

Every one of these changes made the backtest look worse and the live results better. That's the uncomfortable part. A change that improves your backtest deserves suspicion, and a change that hurts it but survives out of sample is usually the real thing.

It's also why we publish every bet before kick-off and never touch the record afterwards. A timestamped public log can't be tuned after the fact, which is more than any backtest can claim. The cost is that the bad weeks sit there in the open, and we think that's a fair price.