Confidence calibration for traders: a practical guide

Confidence calibration is the relationship between how certain you say you are and how often the forecast occurs. It matters because position size usually rises with conviction. If 90% ideas win 55% of the time, the problem is not merely language; the risk budget is being assigned on false premises.

Blind, repeated forecasts create a clean calibration record. Seeing the ticker or news first contaminates the estimate with hindsight and familiarity.

Use a small probability vocabulary

Start with a few stable levels such as 55%, 70% and 90%. Too many choices create fake precision. Define each level in advance and use it across setups so results can be grouped meaningfully.

Score every forecast

Record the probability before the outcome and use a proper scoring rule such as Brier score. Do not delete ambiguous calls or relabel confidence after the fact. Selective memory is one of the main sources of apparent trading skill.

Inspect reliability and discrimination

Calibration asks whether 70% forecasts occur roughly 70% of the time. Discrimination asks whether higher-confidence groups outperform lower-confidence ones. A forecaster can be conservatively calibrated yet unable to rank strong ideas above weak ones.

Change sizing after evidence

If high-confidence calls are not distinct, compress position sizes before searching for a new indicator. Raise risk only when an out-of-sample record shows both calibration and useful separation between confidence bands.

Questions