e12e

How large are large language models? (2025)

This aims to be factual information about the size of large language models. None of this document was written by AI. I do not include any information from leaks or rumors. The focus of this document is on base models (the raw text continuation engines, not 'helpful chatbot/assistants'). This is a view from a few years ago to today of one very tiny fraction of the larger LLM story that's happening.

History

GPT-2,-medium,-large,-xl (2019): 137M, 380M, 812M, 1.61B. Source: openai-community/gpt2. Trained on the unreleased WebText dataset said to 40GB of Internet text - I estimate that to be roughly 10B tokens. You can see a list of the websites that went into that data set here domains.txt.
GPT-3 aka davinci, davinci-002 (2020): 175B parameters. There is a good breakdown of how those parameters are 'spent' here [How d

Linux on the Zenbook 14 OLED (refresh) - Model UM3402YAR

BIOS

The latest BIOS is version 303 "UM3402YAR.303" (Release Date: 09/05/2023). All items posted here will be specific to the model number and BIOS version described here (BIOS 300, 302 were known working). Some things might be specific to Arch Linux but should be generic and portable to other distributions. There are no guarantees and should you wish to try any of this out it is assumed to be at your own risk.

Sound

The sound does not work out of the box (at least not on Arch but I am guessing for most Distros as well).

The patch 0001-ALSA-hda-realtek-Add-quirk-for-ASUS-UM3402YAR-using-.patch is required to be applied to kernels older than v6.4 to ensure that the Realtek sound device is properly bound to the Cirrus amplifiers. If you are using kernel v6.4 or newer this patch is not needed.

The ssdt_csc3551.dsl file can be used per the instructions found in the file header. No need to patch the DSDT, this is a stand-alone SSDT that is easily u

Title

Problem description

We must explain the problem clearly and identify additional details that the team needs to know. We must here describe the context, what we did so far, and the current state.

The description also serves as a trail that we can go back to in the future to understand the reasoning we had at the time and see what restrictions and requirements have changed.

	# put in /etc/systemd/system/netns-wg.service

	[Unit]
	Description=Start a VPN Network Namespace
	After=network.target

	[Service]
	Type=oneshot
	RemainAfterExit=true

	# A Pipeline extension to process steps concurrently
	# Example
	# class ConcurrentPipeline < Pipeline
	# include ConcurrentProcessing
	# end
	#
	# MyPipeline = ConcurrentPipeline.new do \|pl\|
	# pl.step ValidateInput
	#
	# # These steps run concurrently

	from marvin import ai_fn

	@ai_fn
	def generate_people(n: int) -> list[dict]:
	"""Generates a list of n people with random names and ages."""

	generate_people(n=2) # [{'name': 'Olivia', 'age': 30}, {'name': 'Ethan', 'age': 22}]

	# apt install postgresql-client
	# apt-get -y install bash-completion wget
	# wget --no-check-certificate --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc \| apt-key add -
	# echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" \| sudo tee /etc/apt/sources.list.d/pgdg.list
	# apt-get update
	# apt-get -y install postgresql-client-12

	namespace :db do

	desc "Dumps the database to backups"

	#!/bin/bash
	# tailscale completion -- shell-script --

	_tailscale()
	{
	local cur prev words cword
	_init_completion -n = \|\| return

	if [[ $cword -eq 1 ]]; then
	SUBCOMMANDS=$(tailscale --help 2>&1 \| awk '/SUBCOMMANDS/{ f = 1; next } /FLAGS/{ f = 0 } f{print $1}')

	class Whereable
	def initialize(where:, model: Item, ranking_conditions: [], valid: true, data_source: nil)
	@model = model
	@where = where
	@data_source = data_source
	@ranking_conditions = ranking_conditions
	@valid = valid
	end

	def valid?

	{
	"title": "Remap Useless Option Symbol Keys",
	"rules": [
	{
	"description": "Remap Option-G to Option-Command-G",
	"manipulators": [
	{
	"type": "basic",
	"from": {
	"key_code": "g",