Brier score for trading predictions, explained

Brier score turns an UP or DOWN forecast into something more informative than a win rate. For a binary event, convert confidence to a probability and square the difference between that probability and the outcome, where the outcome is 1 if the forecasted event occurred and 0 if it did not.

A perfect forecast scores 0. A smaller score is better. The measure rewards probability estimates that are both accurate and appropriately cautious.

A simple example

Suppose you predict UP with 70% confidence and the stock rises. The score is (0.70 − 1)² = 0.09. If it falls, the score is (0.70 − 0)² = 0.49. The same wrong direction at 55% confidence scores 0.3025, so certainty carries a clear cost.

Scoring DOWN calls

You can define the event as ‘my chosen direction occurs’. Then confidence is always the probability assigned to your call and the result is 1 for correct or 0 for wrong. This keeps the interface intuitive while preserving the mathematics.

What Brier score reveals

Average score combines calibration and discrimination. Decomposing results by confidence band shows more: whether high-confidence calls win more often, and whether each group's hit rate resembles its stated probability.

What it does not measure

Brier score does not include payoff size, transaction costs or correlation between positions. Read the Tape therefore shows paper P&L alongside it. The two answer different questions: forecast quality and the sizing consequence of that forecast.

Questions