brito brito

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

Programming as Theory Building

Peter Naur

Peter Naur's classic 1985 essay "Programming as Theory Building" argues that a program is not its source code. A program is a shared mental construct (he uses the word theory) that lives in the minds of the people who work on it. If you lose the people, you lose the program. The code is merely a written representation of the program, and it's lossy, so you can't reconstruct

	# To run you'll need some secrets:
	# 1. SERPAPI_API_KEY secret in env var - get from https://serpapi.com/
	# 2. OPENAI_API_KEY secret in env var - get from https://openai.com

	import streamlit as st
	import json, os
	from langchain.prompts import PromptTemplate
	from langchain.llms import OpenAI
	from serpapi import GoogleSearch

	-- since it uses dblink it should be enabled in the database.
	-- CREATE EXTENSION dblink;

	-- And you'll may need to grant permissions to use it to your user.
	-- GRANT EXECUTE ON FUNCTION dblink_connect_u(text) TO user;
	-- GRANT EXECUTE ON FUNCTION dblink_connect_u(text, text) TO user;

	-- Usage example:
	-- select g_parsel('insert into osm_polygon_extra select osm_id, st_pointonsurface( st_collect( geom ) ) from osm_polygons group by osm_id;', 'osm_polygons', 12);

	function __SMOKE_TEST__Catalog(){ console.log(
	Catalog([{a:1, b:2}, {b:4, c:7, d:0}, {a:0, b:2}])
	)}
	/*
	┌─┐┌─┐┌┬┐┌─┐┬ ┌─┐┌─┐
	│ ├─┤ │ ├─┤│ │ ││ ┬
	└─┘┴ ┴ ┴ ┴ ┴┴─┘└─┘└─┘ 18w40*/
	function Catalog(items){
	return ([].concat(arguments)).reduce(catalog, {});

	WITH params AS (
	-- Parameters for down stream queries
	SELECT
	15 AS bucket_count,
	80 AS max_bars
	),
	numbers AS (
	-- Change this query to select real data.
	-- For now we make random set of numbers.
	SELECT

	// Shannon entropy
	const entropy = str => {
	return [...new Set(str)]
	.map(chr => {
	return str.match(new RegExp(chr, 'g')).length;
	})
	.reduce((sum, frequency) => {
	let p = frequency / str.length;
	return sum + p * Math.log2(1 / p);
	}, 0);

	<?xml version='1.0' encoding='UTF-8'?>
	<!--
	City of Los Angeles Template
	July 7, 2015
	Oracle WebCenter Portal Template
	-->
	<jsp:root xmlns:jsp="http://java.sun.com/JSP/Page" version="2.1"
	xmlns:f="http://java.sun.com/jsf/core"
	xmlns:c="http://java.sun.com/jsp/jstl/core"
	xmlns:h="http://java.sun.com/jsf/html"