Blog

The capstone of the search-and-overfit arc, and it ends in a negative result — the correct one. A single-symbol dual-timeframe search on ETHUSDT found a config worth +16.35% out-of-sample and +2.62% on an untouched holdout; the Deflated Sharpe Ratio, accounting for ~37,000 trials, deflated it to 0.00. A cross-instrument pass over five majors (ETH/BTC/SOL/BNB/XRP, ~1.18M 1m bars each), selecting by median out-of-sample, kills it for good: dual DSR 0.24 / PBO 0.264, triple DSR 0.14 / PBO 0.327 — both fail the gates. The champion is profitable on 1 of 5 symbols and negative on the rest. This is what the anti-overfit apparatus is for: to stop you from shipping the best of noise as alpha.

Multi-timeframe backtests leak the future through a forming higher-timeframe bar whose final close does not exist yet. You cannot code-review your way to confidence — you have to test it. We reproduce the live bot's closed-bar rule exactly, then prove no leakage with a shifted-future probe: perturb every future bar and assert every past signal and trade is bitwise unchanged. 25/25 parity checks, and the probe has teeth.

The GPU's lead over CPU grows with batch size — 54.5x at one combo per call up to 359.6x at 61 on our multi-timeframe indicator precompute — because a small sweep cannot amortize kernel-launch and transfer overhead. We decompose a headline 167x into a 27x algorithmic win that also helps the CPU and a 6.2x hardware win, show the true GPU-vs-best-CPU lead is only 3.2x single-timeframe and 6.2x multi, and give a decision guide for how wide a sweep must be before a GPU is worth buying into.

Apple's Metal GPU has no float64. Port a vectorized backtest to it naively and the tempting prefix-sum WMA overflows fp32 — max relative error 211× — yet it still runs and returns plausible-looking numbers. The fix is not more precision; it is a different formulation: a direct windowed convolution, fp32-safe to 8×10⁻⁷ and 55.9× faster than single-thread numba. The trap, the arithmetic, and how to prove you didn't fall in.

Drill-down / multi-fidelity search (ASHA, successive halving, Hyperband) screens thousands of configs cheaply and promotes only survivors to the expensive full evaluation. It is a genuine speedup — but it collapses silently if the low-fidelity ranking disagrees with the high-fidelity one. We measured the fold-rank correlation: at one fold Spearman ρ can be 0.03 (ranks almost randomly), climbing to 0.43, 0.67, 0.78, 0.91 as folds accumulate. The fix is one mandatory gate — measure ρ(cheap, full) first, and auto-raise the minimum fidelity to the first rung where ρ ≥ 0.5.

When one backtest is cheap, dumb scrambled Sobol wins on raw throughput — smart samplers (TPE, CMA-ES, ASHA) pay a Python ask/tell tax that drops them 20x, so they evaluate far fewer points at equal wall-clock and lose. Make each eval expensive (multi-TF + walk-forward folds) and the crossover flips. We measured both regimes, and why fold-rank fidelity (ρ@1 rising 0.03→0.43) is the precondition for pruning to pay off.

Not all parameters cost the same to search. A strategy's parameters split into an expensive axis (indicators — recomputed over the whole series) and a cheap axis (decision thresholds — an O(n) pass over precomputed signals). Because indicators are invariant to thresholds, you compute them once and sweep thousands of threshold configs at ~5,600 cfg/s — roughly 1,600x cheaper than recomputing per config. A re-pricing of the curse of dimensionality.