Trading calibration test: does your confidence match your results?

A trading calibration test asks a different question from a profit leaderboard. When you say a setup is highly likely, how often are you actually right? Traders can achieve a tolerable hit rate and still destroy a book by assigning too much confidence to weak forecasts.

Read the Tape records direction and confidence on the same blind daily charts for everyone. The Brier score grades the probability, while paper P&L makes the confidence choice economically visible.

Calibration versus accuracy

Accuracy counts correct directions. Calibration groups comparable confidence statements and checks their realised frequency. If calls labelled near 70% win only half the time, the directional instinct may be ordinary but the confidence process is clearly overstated.

Why Brier score

For a binary outcome, Brier score is the squared gap between forecast probability and the result. Lower is better. A confident mistake receives a much larger penalty than a cautious mistake, while a confident correct call improves the score.

Why attach a paper stake

Probabilities can feel abstract. Read the Tape maps confidence above 50% to a fraction of the book: a small lean takes modest risk and maximum confidence goes all in. This is paper money, but it exposes the sizing consequence of certainty.

How much data is enough

Five or ten calls tell almost nothing. Review confidence bands over dozens of forecasts and across different regimes. The goal is not to force every band to an exact percentage, but to discover systematic overconfidence and situations where discrimination improves.

Questions