Skip to content

Instantly share code, notes, and snippets.

@jonasvanderhaegen
Created August 2, 2026 13:49
Show Gist options
  • Select an option

  • Save jonasvanderhaegen/6bcee7363bf603f4886c1596954e15a6 to your computer and use it in GitHub Desktop.

Select an option

Save jonasvanderhaegen/6bcee7363bf603f4886c1596954e15a6 to your computer and use it in GitHub Desktop.
Sourcing deep 1-minute futures history for backtesting: dxLink depth cap measurement + provider comparison (2026-08-02)

Sourcing deep 1-minute futures history for backtesting: findings

Date: 2026-08-02. Context: an intraday futures strategy pipeline (CME micro contracts, 1m bars) needed more than the ~1 week of history its live feed retains, to enable out-of-sample backtesting. These are the measured findings and the sourcing decision.

TL;DR

  1. The tastytrade/dxFeed dxLink candle service is capped at ~8000 events per snapshot subscription, and windowed re-requests do not reach past it. Effective free depth: about one week of 1m bars per symbol, regardless of how far back you ask.
  2. Cross-instrument concatenation ("stitch 8 roots into one long series") is tempting and wrong: it manufactures bar count, not information. Ratio-adjusted stitching is only legitimate within one root across contract rolls.
  3. For real depth, Databento's GLBX.MDP3 (official CME distributor) is the standout: 16+ years, all four CME exchanges, OHLCV-1m is the cheap schema, and the $125 new-account credit likely covers a multi-root multi-year purchase outright.

Finding 1: the dxLink candle depth cap, measured

A managed backfill job requested 180 days of 1m candles for /MES:XCME (continuous) from the production dxLink feed, with windowed snapshot fetches:

requested_from: 2026-02-03  (epoch ms 1770126349797)
achieved_from:  2026-07-24T02:40Z
events_total:   8001
snapshot_capped: true
windows_fetched: 2
new_rows_total: 0   (the service returned the same week already held)

The cap is event-count-based, not time-based: 8000 events of 1m is roughly one trading week for a near-24h CME contract. Asking with older from-times does not unlock older data. If your pipeline rides tastytrade's dxLink, plan on the feed being a live/recent source only and archive bars yourself from day one.

Finding 2: what does NOT create more history

  • Concatenating different roots into one synthetic series (even with DJ-style ratio matching at the seams so price levels align) fixes level continuity but not the rest: percent volatility is scale-invariant, so the stitched series jumps vol regime at every seam; the synthetic instrument has no meaningful point value, so P&L becomes fiction; and eight instruments over the same calendar week are heavily correlated, so the effective sample barely grows. It can serve as a stress harness for adaptivity, never as promotion evidence.
  • The legitimate version of the same instinct: pool TRADES from per-root runs (native prices, per-root contract multipliers) for statistics, and ratio back-adjust within one root at quarterly rolls, which is exactly how continuous futures contracts are built.
  • The actual fix is depth: sequential real history across regimes, which also unlocks train/validate splits so a strategy is not graded on the week it was fitted.

Finding 3: provider landscape for 1m CME futures history (researched 2026-08-02)

Provider 1m depth Cost shape Notes
Databento (GLBX.MDP3) 16+ years, CME+CBOT+NYMEX+COMEX usage-based $/GB; OHLCV bars are tiny; $125 free credit per new team (6-month expiry) official CME distributor; continuous-contract symbology available; best provenance
FirstRate Data 19y ES, 7y MES one-time bundle + $99.95/yr updates cheapest bulk per dollar; CSV
Kibot free sample: most recent 3 months of 1m; paid: a decade+ free sample, then one-time genuinely free 90-day bootstrap
Barchart Premier ~10 years retail subscription fine, dominated by the two above for this use
dxFeed/dxLink (tastytrade) ~8000 events (~1 week) free, already integrated measured above; live/recent only

Decision

Buy from Databento (OHLCV-1m, continuous symbology, all traded roots, multi-year), expecting the free credit to cover it; keep dxLink for live/recent bars; keep archiving daily captures so the local store compounds. Import path: DBN/CSV into the existing candle store, with roll handling delegated to the vendor's continuous series.

Method note

The evidence ladder that motivated all of this: in-sample backtests (weakest; the strategy grades its own homework) < out-of-sample backtests on deep history (fast, real evidence) < paper trading (future data is unfittable by definition; slowest, strongest pre-live). Depth purchases move you from the first rung to the second; only calendar time buys the third.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment