Skip to content

Instantly share code, notes, and snippets.

@bleonard
Created April 2, 2026 17:00
Show Gist options
  • Select an option

  • Save bleonard/cae5c8f027b486303b443ab6c6c08f13 to your computer and use it in GitHub Desktop.

Select an option

Save bleonard/cae5c8f027b486303b443ab6c6c08f13 to your computer and use it in GitHub Desktop.
Globe Theatre Wikipedia HTML token count: 65,875 tokens (gpt-5 / o200k_base)
import { encoding_for_model, TiktokenModel } from "tiktoken";
const URL = "https://en.wikipedia.org/wiki/Globe_Theatre";
const MODEL = "gpt-5" as TiktokenModel;
const response = await fetch(URL);
const html = await response.text();
const encoding = encoding_for_model(MODEL);
const tokens = encoding.encode(html);
console.log(`HTML length: ${html.length.toLocaleString()} characters`);
console.log(`Token count (${MODEL}): ${tokens.length.toLocaleString()}`);
encoding.free();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment