Skip to content

Instantly share code, notes, and snippets.

@databento-bot
databento-bot / filter_csv.py
Last active August 18, 2025 01:11
Filter for specific symbol on Databento CSV data - Using pandas vs. polars vs. bash
# Side-by-side comparison of pandas vs. polars syntax for reading CSV and filtering by symbol
import pandas as pd
import polars as pl
CSVFILE = "glbx-mdp3-20250716-20250815.ohlcv-1m.csv.zst"
# Using pandas
df = pd.read_csv(CSVFILE)
@databento-bot
databento-bot / databento.csv
Last active July 15, 2025 00:25
OPRA US options data: Comparison of Databento CMBP-1 vs. other vendor quotes
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 18 columns, instead of 15 in line 6.
ts_recv,ts_event,rtype,publisher_id,instrument_id,action,side,price,size,flags,ts_in_delta,bid_px_00,ask_px_00,bid_sz_00,ask_sz_00,bid_pb_00,ask_pb_00,symbol
2023-03-28 09:30:00.067424405-04:00,2023-03-28 09:30:00.067217664-04:00,177,30,654313225,A,N,326.77,1,194,0,321.77,326.77,1,1,0,0,SPY 241220P00720000
2023-03-28 09:30:00.182559853-04:00,2023-03-28 09:30:00.182352128-04:00,177,30,654313225,A,N,,0,194,0,,,0,0,0,0,SPY 241220P00720000
2023-03-28 09:30:00.463387013-04:00,2023-03-28 09:30:00.463180032-04:00,177,30,654313225,A,N,329.22,1,194,0,319.22,329.22,1,1,0,0,SPY 241220P00720000
2023-03-28 09:30:00.536456272-04:00,2023-03-28 09:30:00.536249088-04:00,177,30,654313225,A,N,328.28,1,194,0,319.22,328.28,1,1,0,0,SPY 241220P00720000
2023-03-28 09:30:00.537369130-04:00,2023-03-28 09:30:00.537161984-04:00,177,30,654313225,A,N,318.28,1,194,0,318.28,328.28,1,1,0,0,SPY 241220P00720000
2023-03-28 09:30:00.661258449-04:00,2023-03-28 09:30:00.661050624-04:00,177,30,654313225,A,N,328.28,1,194,0,318.28,328.28,2,
@databento-bot
databento-bot / interpolate_ohlcv.py
Last active July 6, 2025 07:23
Interpolate OHLCV-1s on Databento to ensure exactly one row per second
"""
Databento follows the standard convention of only printing OHLCV records when
trades actually occur within the interval. If no trade occurs within the interval,
no record is printed.
This approach is adopted by most data vendors for two key reasons:
1. Multiple interpolation strategies exist and the optimal choice depends on the
specific use case. Client-side interpolation keeps the strategy transparent.
2. This reduces storage and bandwidth requirements, especially for markets with
many illiquid instruments like options.
@databento-bot
databento-bot / resample_ohlcv.py
Created July 6, 2025 06:26
Resample 1-minute OHLCV bars to 5-minute OHLCV bars on Databento
import pandas as pd
import databento as db
client = db.Historical()
data = client.timeseries.get_range(
dataset="XNAS.ITCH",
schema="ohlcv-1m",
symbols="AAPL",
@databento-bot
databento-bot / ng_summary.py
Created July 6, 2025 04:40
Get trade counts on all listed NG outrights
import databento as db
import pandas as pd
client = db.Historical()
def print_ng_summary_by_date(date_str: str):
date = pd.to_datetime(date_str)
weekday = date.weekday()
ts_event rtype publisher_id instrument_id action side price size channel_id order_id flags ts_in_delta sequence symbol
ts_recv
2025-04-25 13:30:00.002415517+00:00 2025-04-25 13:30:00.000435143+00:00 160 1 42003627 F A 5510.25 2 8 6863196218407 0 14070 333790855 MESM5
2025-04-25 13:30:00.002415517+00:00 2025-04-25 13:30:00.000435143+00:00 160 1 42003627 F A 5510.25 1 8 6863196218557 0 14070 333790855 MESM5
2025-04-25 13:30:00.002415517+00:00 2025-04-25 13:30:00.000435143+00:00 160 1 42003627 F A 5510.50 1 8 6863196218273 0 14070 333790855 MESM5
2025-04-25 13:30:00.002495446+00:00 2025-04-25 13:30:00.000756343+00:00 160 1 42003627 F A 5510.50 1 8 6863196218273 0 12976 333790857 MESM5
2025-04-25 13:30
@databento-bot
databento-bot / scanner.py
Last active July 27, 2025 08:57
Real-time scanner for detecting large price movements in all US stocks and ETFs
"""
This script demonstrates how to build a real-time scanner that detects significant
price movements across all US stocks and ETFs using Databento's market data APIs.
Features:
- Handles entire US equities universe of ~9,000 symbols efficiently
- Median sub-ms feed delay to NY4/5 (WAN-shaped) and ~5s to start scanning
- Monitors all US stocks and ETFs for price movements exceeding a configurable threshold
- Compares current prices against previous day's closing prices
- Displays alerts when significant moves are detected
@databento-bot
databento-bot / resample.py
Created April 23, 2025 01:15
Resample OHLCV-1s with uniform
"""
OHLCV-1s data follows standard convention of most vendors and doesn't print on intervals
when there's no trade. This is the typical approach as there's no way to infer the type
of price interpolation or forward fill preferred. If you need prices printed uniformly
with forward filling or price interpolation, you can follow this example to do it on
client side.
See also: https://databento.com/docs/examples/basics-historical/tick-resampling/example
"""
// Condenses MEMX notices on https://info.memxtrading.com/category/alerts-notices/
const s=document.createElement("style");s.innerHTML="*{margin:0!important;padding:0!important;line-height:1!important;font-size:10px!important;letter-spacing:0!important}div,p,span,a,li,td,th{line-height:1!important;font-size:10px!important}table{border-spacing:0!important;border-collapse:collapse!important}",document.head.appendChild(s),(async()=>{const e=document.querySelector("main");if(!e)return void console.error("No <main> element found.");document.querySelectorAll("div.nav-links").forEach(t=>t.remove());for(let t=2;t<=91;t++){const n=`https://info.memxtrading.com/category/alerts-notices/page/${t}/`;try{const t=await fetch(n),o=await t.text(),i=new DOMParser().parseFromString(o,"text/html");i.querySelectorAll("hr, article").forEach(t=>e.appendChild(t.cloneNode(!0))),console.log(`Appended page ${t}`)}catch(a){console.error(`Error fetching page ${t}:`,a)}}})();
@databento-bot
databento-bot / compare_cme_eod.py
Created March 3, 2025 05:18
Compare Databento vs. CME end-of-day settlement volumes
from enum import Enum
import databento as db
import pandas as pd
class Stat(Enum):
SETTLEMENT_VOLUME = 6
client = db.Historical()