Blog

Multi-timeframe backtests leak the future through a forming higher-timeframe bar whose final close does not exist yet. You cannot code-review your way to confidence — you have to test it. We reproduce the live bot's closed-bar rule exactly, then prove no leakage with a shifted-future probe: perturb every future bar and assert every past signal and trade is bitwise unchanged. 25/25 parity checks, and the probe has teeth.

The GPU's lead over CPU grows with batch size — 54.5x at one combo per call up to 359.6x at 61 on our multi-timeframe indicator precompute — because a small sweep cannot amortize kernel-launch and transfer overhead. We decompose a headline 167x into a 27x algorithmic win that also helps the CPU and a 6.2x hardware win, show the true GPU-vs-best-CPU lead is only 3.2x single-timeframe and 6.2x multi, and give a decision guide for how wide a sweep must be before a GPU is worth buying into.

Apple's Metal GPU has no float64. Port a vectorized backtest to it naively and the tempting prefix-sum WMA overflows fp32 — max relative error 211× — yet it still runs and returns plausible-looking numbers. The fix is not more precision; it is a different formulation: a direct windowed convolution, fp32-safe to 8×10⁻⁷ and 55.9× faster than single-thread numba. The trap, the arithmetic, and how to prove you didn't fall in.

Drill-down / multi-fidelity search (ASHA, successive halving, Hyperband) screens thousands of configs cheaply and promotes only survivors to the expensive full evaluation. It is a genuine speedup — but it collapses silently if the low-fidelity ranking disagrees with the high-fidelity one. We measured the fold-rank correlation: at one fold Spearman ρ can be 0.03 (ranks almost randomly), climbing to 0.43, 0.67, 0.78, 0.91 as folds accumulate. The fix is one mandatory gate — measure ρ(cheap, full) first, and auto-raise the minimum fidelity to the first rung where ρ ≥ 0.5.

When one backtest is cheap, dumb scrambled Sobol wins on raw throughput — smart samplers (TPE, CMA-ES, ASHA) pay a Python ask/tell tax that drops them 20x, so they evaluate far fewer points at equal wall-clock and lose. Make each eval expensive (multi-TF + walk-forward folds) and the crossover flips. We measured both regimes, and why fold-rank fidelity (ρ@1 rising 0.03→0.43) is the precondition for pruning to pay off.

Not all parameters cost the same to search. A strategy's parameters split into an expensive axis (indicators — recomputed over the whole series) and a cheap axis (decision thresholds — an O(n) pass over precomputed signals). Because indicators are invariant to thresholds, you compute them once and sweep thousands of threshold configs at ~5,600 cfg/s — roughly 1,600x cheaper than recomputing per config. A re-pricing of the curse of dimensionality.

We benchmarked eight backtest engines on one identical parameter sweep — 150k bars, 80 HMA-cross combos, trade-count parity locked at 2707. Two of the most popular event-driven frameworks came in slower than a hand-written pandas loop, while a vectorized/compiled engine ran the same work ~13,000× faster. A study of the per-bar overhead that popular libraries were never built to amortize.