A Rigorous Futures Backtesting Checklist: From Data Audit to Out-of-Sample Test

Write the hypothesis before measuring performance

State the economic or market mechanism, the expected direction of the effect, and the conditions that should weaken it. “Momentum works” is not a testable protocol. Specify instrument universe, dated-contract eligibility, signal inputs, decision frequency, holding horizon, position limits, roll treatment, and the execution event that follows each decision. Define the primary metric and acceptable drawdown or tail-risk limits before viewing results.

Separate exploratory questions from confirmatory tests. Exploration can generate hypotheses; it cannot also supply an untouched final evaluation. Record known event exclusions and whether they were chosen before or after seeing performance. If a strategy claims capacity, add a participation constraint and explain which observed activity measure supports it.

Finally, list failure conditions: a signal that reverses under a small timing change, disappears after plausible costs, or depends on one contract year may not merit deployment research. These criteria are research controls, not a prediction of future returns. Keep the protocol under version control so later amendments remain visible.

Inventory the dataset and its transformations

Create a manifest covering every input file, product, dated contract or continuous series, date range, schema version, interval, checksum, and row count. Note gaps in coverage and differences among instruments. Record which files are derived datasets and how they relate: do not silently combine raw-price and adjusted-price fields or mix a daily session bar with UTC-midnight calendar days.

For these per-contract tick derived datasets, timestamps ts and recv_time are UTC and equal, event_time is null, price is float64, and volume is uint32. bid_sz and ask_sz represent executed volume at bid and ask, not resting book sizes; oi is zero because open interest is unavailable. These constraints must shape the research design—queue models and open-interest filters cannot be inferred from the files.

Continuous derived data add price_raw and back_adjustment, with price = price_raw + back_adjustment. Bars are derived from back-adjusted continuous ticks. Their columns are ts/open/high/low/close/volume/trade_count, at 10s, 1m, 2m, 5m, 30m, 1h, 4h, and 1D. Record that intraday bars are epoch-aligned UTC, daily bars follow Chicago 17:00 sessions with DST, and empty buckets are omitted. Consult the documentation rather than relying on this summary as a substitute for the schema.

Validate schema, ordering, and session alignment

Load with explicit dtypes and timezone handling. Reject unexpected columns, nulls in required fields, non-finite prices, invalid timestamps, and numeric overflow; do not let a parser silently turn malformed values into missing data. Verify ts == recv_time where that invariant applies, and test that timestamps are monotonic within the intended contract and file partition. Define how ties are ordered because equal timestamps do not establish a causal sequence among trades.

Recompute continuous identities on samples: adjusted price minus raw price must equal the stored adjustment. For bars, check low ≤ open, close ≤ high, nonnegative volume, and trade-count consistency. Resample a sample of tick intervals independently and compare open, high, low, close, volume, and count. Confirm that 1D boundaries follow the exchange-local session and that intraday epoch alignment does not drift after a DST change.

Unit-test calendar utilities around 17:00 Chicago, both DST transitions, weekends, and early closes. A correct UTC sort can still produce an incorrect session feature. Keep the failed-row examples and validation report as artifacts, not just a final pass count.

Investigate missingness, duplicates, and market microstructure

Duplicate tests need a defined key. Exact duplicate rows, repeated timestamp-price-volume combinations, and legitimate simultaneous trades are different cases. Quantify each before removal and preserve an audit trail. Deduplicating by timestamp alone can destroy valid activity. Similarly, missing bars are expected for empty buckets; distinguish inactivity and scheduled closures from unexplained gaps using the product calendar.

Test prices against the instrument’s tick grid with a tolerance appropriate to binary floating-point representation. Investigate negative or zero prices where the product rules do not permit them, abrupt scale changes, impossible jumps, and volume anomalies. Do not “clean” valid extreme prints merely because they hurt a strategy. Compare suspect observations across related intervals and dated contracts, then classify each resolution.

validation_report = {
    "rows": rows_checked,
    "duplicate_rows": exact_duplicates,
    "nonmonotonic_contracts": ordering_failures,
    "off_grid_prices": grid_failures,
    "unexplained_session_gaps": gap_failures,
}

Because bid/ask size fields are executed volumes, use them only for suitable trade-side diagnostics. Their values say nothing about displayed depth, queue position, or the amount available for a later market order.

Audit continuous rolls as strategy decisions

A continuous file answers a signal-history question; it does not remove the need to simulate transitions between dated contracts. Identify the roll policy, when its inputs become known, and the source and destination contracts. Reconstruct positions through each switch and charge transactions on both sides. Check whether volume-based triggers use only information available at the decision timestamp.

Back adjustment preserves a chosen historical price relationship by adding back_adjustment to price_raw. It can create adjusted levels that were never quoted, including negative values in some settings. Compute signals consistently on the intended adjusted series, but model order prices, contract multipliers, and cash P&L on the appropriate dated-contract economics. Never interpret the additive adjustment itself as realized income.

Test alternate plausible roll windows without selecting the one that maximizes final profit. Report performance by roll episode and inspect whether returns cluster around transitions. For a fuller treatment of timing errors, see avoiding look-ahead bias in futures research.

Make execution and costs adversarial

Specify order type, submission delay, eligible session, fill price, partial-fill behavior, and cancellation logic. A signal computed from a bar close cannot normally fill at that same close without an explicit, defensible timing mechanism. Reject orders during closures and handle the next opening according to the order instructions. During a roll, model both contracts and their separate liquidity.

Include commissions, exchange and clearing charges where applicable, bid–ask spread, slippage, and market impact. Vary costs with volatility, activity, time of day, and trade size when evidence supports that structure. Without resting depth, do not claim a precise queue simulation; use conservative fill scenarios and disclose the limitation. Treat zero-volume intervals as no demonstrated execution opportunity rather than automatic fills at a carried-forward price.

Report gross and net results, cost per trade, turnover, participation, and sensitivity to delayed entry. Compare immediate, next-eligible, and adversely priced fills. The companion guide to futures backtest costs and slippage provides a framework for these assumptions. No cost model guarantees live outcomes; its purpose is to expose dependence on unrealistic execution.

Freeze chronological development and final evaluation

Split time into training, validation, and a final test period before model fitting. Fit scalers, thresholds, universe rules, and feature selection on training data only. Use validation for model choice and parameter selection; use the test once for the prespecified final evaluation. If test results motivate changes, that period has entered development and a new untouched evaluation is required.

When labels or positions overlap across a boundary, use purging and an appropriate embargo so future information does not leak through adjacent samples. Walk-forward evaluation should repeat the complete fitting and selection process at each step, including roll-rule calibration if it is learned. Keep contract regimes, volatility environments, and holiday periods represented deliberately rather than relying on a convenient random split.

Assess parameter neighborhoods, not only the optimum. Heat maps or response tables should show whether nearby lookbacks, thresholds, and holding periods behave coherently. Compare against simple benchmarks and a no-trade baseline. Report dispersion across instruments and time blocks; one exceptional year is not robustness.

Stress scenarios and preserve reproducibility

Run sensitivity tests for one-bar and multi-bar delays, wider spreads, multiplied costs, reduced participation, alternate rolls, missing sessions, and opening gaps. Include historical stress windows and synthetic shocks that break the strategy’s assumed mechanism. Evaluate what happens when volatility doubles, correlations converge, or the signal arrives late. Show where the strategy stops trading or breaches risk limits.

Make the run reconstructable: code revision, environment specification, configuration, random seed, data manifest and checksums, calendar version, roll schedule, validation outputs, and generated metrics should share a run identifier. A deterministic rerun should match the recorded outputs. Save rejected configurations as well as the selected one to reveal selection intensity.

Sign-off checklist

• Hypothesis, primary metric, failure criteria, and splits predate final evaluation.
• Schema, dtypes, UTC ordering, duplicates, gaps, tick grid, and bar identities pass documented checks.
• Roll mapping, raw versus adjusted prices, session calendars, and DST handling are verified.
• Fills respect information timing and market hours; costs and liquidity limits are stressed.
• Validation performs model selection; the untouched test is reported without retroactive tuning.
• Parameter neighborhoods, subperiods, adverse scenarios, and benchmarks are disclosed.
• Artifacts, checksums, configuration, and code reproduce the exact report.
• Limitations and negative findings remain in the sign-off record.

Sign-off means the evidence and limitations support the stated research conclusion—not that the backtest predicts or promises future performance.