Dovie Weinstock DovieW

How large are large language models? (2025)

This aims to be factual information about the size of large language models. None of this document was written by AI. I do not include any information from leaks or rumors. The focus of this document is on base models (the raw text continuation engines, not 'helpful chatbot/assistants'). This is a view from a few years ago to today of one very tiny fraction of the larger LLM story that's happening.

History

GPT-2,-medium,-large,-xl (2019): 137M, 380M, 812M, 1.61B. Source: openai-community/gpt2. Trained on the unreleased WebText dataset said to 40GB of Internet text - I estimate that to be roughly 10B tokens. You can see a list of the websites that went into that data set here domains.txt.
GPT-3 aka davinci, davinci-002 (2020): 175B parameters. There is a good breakdown of how those parameters are 'spent' here [How d

You are a teacher of algorithms and data-structures who specializes in the use of the socratic method of teaching concepts. You build up a foundation of understanding with your student as they advance using first principles thinking. Explain the subject that the student provides to you using this approach. By default, do not explain using source code nor artifacts until the student asks for you to do so. Furthermore, do not use analysis tools. Instead, explain concepts in natural language. You are to assume the role of teacher where the teacher asks a leading question to the student. The student thinks and responds. Engage misunderstanding until the student has sufficiently demonstrated that they've corrected their thinking. Continue until the core material of a subject is completely covered. I would benefit most from an explanation style in which you frequently pause to confirm, via asking me test questions, that I've understood your explanations so far. Particularly helpful are test questions related to sim

Prompt for Migrating a Java/Selenium/RestAssured Framework to Playwright/TypeScript

I have a Java-based automation framework using Selenium for UI testing and RestAssured for API testing. The framework's structure and test cases are contained within a directory named [Source Directory Name] at the root level of this project. Please migrate this entire framework to use Playwright with TypeScript for both UI and API testing. The new framework should be placed in a new directory named [Destination Directory Name] at the root level.

The current framework includes:

UI Tests: Implemented using Selenium, likely with a Page Object Model (POM) or similar design pattern. These tests cover various user interactions with the web application.
API Tests: Implemented using RestAssured, covering the testing of RESTful APIs. These tests verify API functionality, data integrity, and response codes.
Test Framework: Likely using JUnit or TestNG for test execution, reporting, and assertion.

👾 Error

fatal: detected dubious ownership in repository at '/storage/emulated/0/<some_folder>'
To add an exception for this directory, call:

        git config --global --add safe.directory /storage/emulated/0/<some_folder>

🤔 Why do I see this error ?

This is a simple check (by Git) comparing current user ID and the ID of the owner of the current folder. If they do not match (in case of rootless Termux, root owns them all.), Git will not permit any action (even git status).

Say you want to listen to youtube videos as a podcast but you're too stubborn to use the AWS instant install thing on the podsync home page.

Before you start: if you're using Pocket Casts you'll want to make sure this all works before adding the podcast. Pocket Casts will cache your podcasts and then if you goofed it will always be referencing the non-working podcast. You can fix that by changing the subdomain you use. But it's just easier if you get it right from the start. You can test that it works by using something like Apple Podcasts, which will not cache and will just work if it works.

I'm kind of expecting that you're comfortable (though not necessarily experienced in) logging into your router, accessing the terminal and doing git commands, managing domain names, and that docker works on your system.

Use a Dynamic DNS service to set up

Connecting to an Android device over SSH

Initial Setup

Install Android App Termux from APKPure or AppStore. If the app exists, just delete and re-install it to get the latest version, The APK can be downloaded from https://apkpure.com/termux/com.termux/ Install the APK using by running

adb install ~/Downloads/Termux_v0.73_apkpure.com.apk

Unofficial Bash Strict Mode

Sometimes a programming language has a "strict mode" to restrict unsafe constructs. E.g., Perl has use strict, Javascript has "use strict", and Visual Basic has Option Strict. But what about bash? Well, bash doesn't have a strict mode as such, but it does have an unofficial strict mode:

set -euo pipefail

set -e

	#!/bin/bash

	# Configuration
	GITEA_REPO="username/repository"
	GITEA_API_TOKEN="your_gitea_api_token"
	GITHUB_REPO="username/repository"
	GITHUB_API_TOKEN="your_github_api_token"
	GITEA_API_URL="https://your_gitea_instance/api/v1"
	GITHUB_API_URL="https://api.github.com"
	ISSUE_FILE="gitea_issues.json"

	$to = "[email protected],[email protected]" # comma separate multiple recipients
	$from = "[email protected]"

	$subject = "subject goes here"
	$body = "message body goes here"

	$smtpServer = "smtp.gmail.com"

	$smtpClient = New-Object Net.Mail.SmtpClient($smtpServer, 587) # port 25 for non SSL, port 587 for SSL
	$smtpClient.EnableSsl = $true # set to $true for SSL

	""""""""""""""""""""""
	" Leader
	""""""""""""""""""""""
	" let mapleader=,
	" can't set leaders in Obsidian vim, so the key just has to be used consistently.
	" However, it needs to be unmapped, to not trigger default behavior: https://github.com/esm7/obsidian-vimrc-support#some-help-with-binding-space-chords-doom-and-spacemacs-fans
	unmap ,

	" map ; to : in normal mode, so that I don’t rely on the shift key
	" nmap ; :