ZY (Jerry) mathsrocks

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

Machine Learning Design Patterns

This book is all about patterns for doing ML. It's broken up into several key parts, building and serving. Both of these are intertwined so it makes sense to read through the whole thing, there are very many good pieces of advice from seasoned professionals. The parts you can safely ignore relate to anything where they specifically use GCP. The other issue with the book it it's very heavily focused on deep learning cases. Not all modeling problems require these. Regardless, let's dive in. I've included the stuff that was relevant to me in the notes.

Most Interesting Bullets:

Machine learning models are not deterministic, so there are a number of ways we deal with them when building software, including setting random seeds in models during training and allowing for stateless functions, freezing layers, checkpointing, and generally making sure that flows are as reproducible as possib

If you are on a Mac, substitute command for control. Don't type the + (it means press both keys at once).

Shortcuts when in either command mode (outside the cells) or edit mode (inside a cell):

Shift + Enter run selected cell or cells - if no cells below, insert a code cell below
Ctrl + B toggle hide/show left sidebar
Ctrl + S save and checkpoint
Ctrl + Shift + S save as

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h or --help depending on your psql version):

-E: will describe the underlaying queries of the \ commands (cool for learning!)
-l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)

	from pymc3.math import log, exp, where
	import pymc3 as pm
	import numpy as np


	# We use the "calibration" portion of the dataset to train the model
	N = rfm_cal_holdout.shape[0] # number of customers
	x = rfm_cal_holdout['frequency_cal'].values # repeat purchase frequency
	t_x = rfm_cal_holdout['recency_cal'].values # recency
	T = rfm_cal_holdout['T_cal'].values # time since first purchase (T)

	'''
	Non-parametric computation of entropy and mutual-information

	Adapted by G Varoquaux for code created by R Brette, itself
	from several papers (see in the code).

	These computations rely on nearest-neighbor statistics
	'''
	import numpy as np