Skip to content

Instantly share code, notes, and snippets.

@bleonard
Created April 2, 2026 17:00
Show Gist options
  • Select an option

  • Save bleonard/e88aa60bf3bb729ca56ce302144d5aa6 to your computer and use it in GitHub Desktop.

Select an option

Save bleonard/e88aa60bf3bb729ca56ce302144d5aa6 to your computer and use it in GitHub Desktop.
Colosseum Wikipedia HTML token count: 118,862 tokens (gpt-5 / o200k_base)
import { encoding_for_model, TiktokenModel } from "tiktoken";
const URL = "https://en.wikipedia.org/wiki/Colosseum";
const MODEL = "gpt-5" as TiktokenModel;
const response = await fetch(URL);
const html = await response.text();
const encoding = encoding_for_model(MODEL);
const tokens = encoding.encode(html);
console.log(`HTML length: ${html.length.toLocaleString()} characters`);
console.log(`Token count (${MODEL}): ${tokens.length.toLocaleString()}`);
encoding.free();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment