Christopher Akiki cakiki

Learning LLMs in 2025

So you know how the transformer works, and you know basic ML/DL, and you want to learn more about LLMs. One way to go is looking into the various "algorithmic" stuff (optimization algorithms, RL, DPO, etc). Lot's of materials on that. But the interesting stuff is (in my opinion at least) not there.

This is an attempt to collect a list of academic (or academic-like) materials that explore LLMs from other directions, and focus on the non-ML-algorithmic aspects.

Courses

David Chiang's Theory of Neural Networks course.
This is not primarily LLMs, but does have substantial section on Transformers. Formal/Theory. More of a book than a course.

	import os
	import sys
	import time
	import math
	import pickle
	from contextlib import nullcontext
	from pathlib import Path
	import subprocess
	from dataclasses import dataclass
	import inspect

	""" Implementation of OKapi BM25 with sklearn's TfidfVectorizer
	Distributed as CC-0 (https://creativecommons.org/publicdomain/zero/1.0/)
	"""

	import numpy as np
	from sklearn.feature_extraction.text import TfidfVectorizer
	from scipy import sparse


	class BM25(object):


	অবশ্য
	অনেক
	অনেকে
	অনেকেই
	অন্তত
	অথবা
	অথচ
	অর্থাত
	অন্য

	# 4x 1 chip (2 cores) per process:
	os.environ["TPU_CHIPS_PER_HOST_BOUNDS"] = "1,1,1"
	os.environ["TPU_HOST_BOUNDS"] = "1,1,1"
	# Different per process:
	os.environ["TPU_VISIBLE_DEVICES"] = "0" # "1", "2", "3"
	# Pick a unique port per process
	os.environ["TPU_MESH_CONTROLLER_ADDRESS"] = "localhost:8476"
	os.environ["TPU_MESH_CONTROLLER_PORT"] = "8476"

	# 1-liner for bash: TPU_CHIPS_PER_HOST_BOUNDS=1,1,1 TPU_HOST_BOUNDS=1,1,1 TPU_VISIBLE_DEVICES=0 TPU_MESH_CONTROLLER_ADDRESS=localhost:8476 TPU_MESH_CONTROLLER_PORT=8476