Paul Furber paulfurber

How large are large language models? (2025)

This aims to be factual information about the size of large language models. None of this document was written by AI. I do not include any information from leaks or rumors. The focus of this document is on base models (the raw text continuation engines, not 'helpful chatbot/assistants'). This is a view from a few years ago to today of one very tiny fraction of the larger LLM story that's happening.

History

GPT-2,-medium,-large,-xl (2019): 137M, 380M, 812M, 1.61B. Source: openai-community/gpt2. Trained on the unreleased WebText dataset said to 40GB of Internet text - I estimate that to be roughly 10B tokens. You can see a list of the websites that went into that data set here domains.txt.
GPT-3 aka davinci, davinci-002 (2020): 175B parameters. There is a good breakdown of how those parameters are 'spent' here [How d

tl;dr;

UPDATE Mon Mar 10 10:51:31 AM EDT 2025 Check out the newer ktransformers guide for how to get it running faster! About 3.5 tok/sec on this same gaming rig. Big thanks to Supreeth Koundinya with analyticsindiamag.com for the article!

You can run the real deal big boi R1 671B locally off a fast NVMe SSD even without enough RAM+VRAM to hold the 212GB dynamically quantized weights. No it is not swap and won't kill your SSD's read/write cycle lifetime. No this is not a distill model. It works fairly well despite quantization (check the unsloth blog for details on how they did that).

The basic idea is that most of the model itself is not loaded into RAM on startup, but mmap'd. Then kv cache will take up some RAM. Most of your system RAM is left available to serve as disk cache for whatever experts/weights are currently most u

PostgreSQL is Enough

Background and Cron Jobs

Open Source and Videogames - Resources

This git include a list of programs, tools, engines and libraries free and open source intended to make videogames.

NOTE: This gist is a support material for the talk "Open Source and Videogames" given by me, Ramon Santamaria, on October 26th 2021 in Canòdrom, Barcelona. All the materials listed here were explained in detail in a +2 hours talk.

How to setup a practically free CDN using Backblaze B2 and Cloudflare

⚠️ Note 2023-01-21
Some things have changed since I originally wrote this in 2016. I have updated a few minor details, and the advice is still broadly the same, but there are some new Cloudflare features you can (and should) take advantage of. In particular, pay attention to Trevor Stevens' comment here from 22 January 2022, and Matt Stenson's useful caching advice. In addition, Backblaze, with whom Cloudflare are a Bandwidth Alliance partner, have published their own guide detailing how to use Cloudflare's Web Workers to cache content from B2 private buckets. That is worth reading,

Time Travel Debugging

Time Travel refers to the ability to record a tab and later replay it ([WebReplay][wrr]). The technology is useful for local development, where you might want to:

pause and step forwards or backwards
pause and rewind to a prior state
rewind to the time a console message was logged
rewind to the time an element had a certain style or layout
rewind to the time a network asset loaded

FFmpeg cheat sheet

A list of useful commands for the FFmpeg command line tool.

Download FFmpeg: https://www.ffmpeg.org/download.html

Full documentation: https://www.ffmpeg.org/ffmpeg.html

Basic conversion

JS First

About This Manifesto

Have you ever argued for or against teaching language X as the first language in a university computer science curriculum? If so, I hope that your arguments:

were first and foremost about students, considering the question “What do we want students to gain from their experience with a first language?”, not “Is language X better than language Y?” because the latter question requires too much context and isn’t really answerable;
kept in mind that ultimately we want to train polyglots, so the first language is never the only language; and
took into account previous work from computing educators, and education theorists and practitioners in general.

	#!/usr/bin/bash -xe

	cat <<EOF > "${HOME}/.config/systemd/user/zoom.slice"
	[Slice]
	AllowedCPUs=0-4
	MemoryHigh=6G
	EOF

	cat /usr/share/applications/Zoom.desktop \| sed -E 's#^(Exec=).*$#Exec=/usr/bin/systemd-run --user --slice=zoom.slice /opt/zoom/ZoomLauncher#' > "${HOME}/.local/share/applications/Zoom.desktop"

	Compiler design
	Compilers
	https://github.com/snazzy-d/SDC
	https://github.com/dbohdan/embedded-scripting-languages
	https://github.com/ishiura-compiler/CF3
	https://gitlab.com/styx-lang/styx
	http://mlton.org/CompilerOverview
	https://github.com/TurkeyMan/mlang
	https://github.com/FeepingCreature/fcc
	https://github.com/jondgoodwin/cone