02.08.2026

Your backtest is a random number generator

Markets|Python|1 min|178 words

Search a large enough space of rules and something will clear any threshold you set. The Sharpe ratio you are admiring is a maximum over many trials, and the distribution of a maximum is not the distribution of a draw.

01SETUP

What the number actually is

If you test mm independent strategies whose true edge is zero, the expected best observed Sharpe grows roughly like 2logm\sqrt{2 \log m}. Twenty variations buys you about 2.4 standard deviations of pure noise.

02REMEDY

The deflated ratio

Bailey and López de Prado's deflated Sharpe adjusts for the number of trials and for the skew and kurtosis of returns. It is not a cure, but it converts a number you cannot interpret into one you can.

pythonanalysis/deflated.py

see Bailey & Lopez de Prado (2014)

from scipy.stats import norm

def deflated_sharpe(sr, n, m, skew, kurt):
    """Probability the observed Sharpe exceeds a zero-skill benchmark."""
    sr0 = expected_max_sharpe(m)
    se = ((1 - skew * sr + (kurt - 1) / 4 * sr**2) / (n - 1)) ** 0.5
    return norm.cdf((sr - sr0) / se)

The honest version of a backtest reports how many things you tried.