,Preci0us.md

preci0us -- one engine, many learners

One ring to rule them all. Classify, regress, cluster, anomaly, sample/synth all = one engine + a (y_fn, reducer) pick. Apps add no logic.

Files

• ,Preci0us.md -- this landing page (design spine)
• ideas.md -- long-form primitives + app reductions

Why it works

Sparsity of Influence. Median intrinsic dim <=5 in >=80% of MOOT tasks; top-5 vars carry 80% of y-variance. BINGO: discretize those few dims into a few buckets, capture y-variation. Data already clumped. Engine = find clumps, read them off. Divide cheap, tally cheap, explain-by-neighbor free.

Grammar (the only types)

Atom    = num | str | nil
Row     = Atom+
Col     = Num | Sym
Num     = (at, txt, n, mu, m2, lo, hi)
Sym     = (at, txt, n, has)
Cols    = (all, x, y)
Data    = (rows, cols)
Cut     = (col, val)
Node    = (data, cut?, left?, right?)
Forest  = Node+

5 shapes of substance: Col(Num|Sym), Data, Cut, Node, Forest.

The engine

engine(d, y_fn, reducer)
  = tree(d, y_fn) >> relevant(row) >> reducer(leaf)

Whole library. Rest = pick y_fn, pick reducer, post-process.

Cost: one bit that matters

* = paid (queries y-label). Else free. Active learning = minimize starred calls. All else ~0 cost.

paid	why
add on y-col	needs label
cuts, tree, forest	call y_fn
peeks	bounded label budget

Functions (simple -> complex)

1. ctors Num.new Sym.new head 2. tally Num.add* Sym.add* Num.sub Sym.sub 3. col lemmas mid spread norm like cut 4. geometry distx disty near far anchors 5. data/io Data.new csv clone 6. splits divides cuts* Cut.new 7. cluster algebra group select describe contrast 8. structure Node.new tree* relevant Forest.new forest* 9. readouts vote mean density likely 10. acquisition random diverse smote kpp peeks*

Apps = (y_fn, reducer), no new code

app	y_fn	reducer
ctree	klass	mode
rtree	disty	mean
lnn-c	row id (depth-1)	mode of k
lnn-r	row id	mean of k
layes	klass	argmax like
lmeans	distx-to-centroid	centroid
lnomaly	none	1/density
fastmap	distx(a)-distx(b)	identity
ftree	distx(a)-distx(b)	mode/mean
peeks	disty + budget	mean

Buse-Zimmermann surface

cell	reduction
Discover	group + describe
Predict	tree + relevant + reducer
Explain	contrast(mine, nearest-diff-y)
Trust	forest agreement (Rashomon |B|)

Pearl's ladder

rung	this stack
1 Association	near like describe regress/classify
2 Intervention	contrast = min x-change flipping y
3 Counterfactual	extrapolate/simulate, only inside |B| hull

Stops at 2.5: no fantasy do-ops outside observed data.

Radical shrink (vs lull/)

delete/move	LOC	why
5 stub app files -> 1 apps.lua table	-160	each = 2-tuple over engine
ftree/poles/sampleCap	-40	ftree = tree w/ anchor-proj y_fn
attachD/_cache/d.x/d.y -> memo(fn)	-25	shape stays (rows, cols)
klass/goal slots	-5	derive from header suffix
Num.bin/Sym.bin	-10	floor(bins*norm) inline
Confuse+stat tests -> eval.lua	-80	orthogonal, outside grammar

~1313 LOC -> ~540 LOC (~60% cut). End state: lib (helpers) + lull (engine + algebra) + apps (10-row table) + eval (metrics, off to the side).

Add to close the grammar

L4 algebra missing: group select describe contrast (~40 LOC). Then layes lmeans explain trends forecast collapse to one-liners.

ideas.md

precious.md — primitives under all apps

Claim: classify, regress, cluster, anomaly, sample/synth all reduce to a small set of primitives. Apps are thin wrappers.

Primitives (in lull.lua)

primitive	sig	purpose
m.distx	cols,r1,r2,p --> qty	row-to-row Minkowski (x cols)
m.near	cols,query,rows,p --> rows	rows sorted asc by distx
m.disty	cols,row,p --> qty	row-to-ideal (y cols, multi-obj)
m.cuts	cols,rows,y,bins,Sumr --> best	best split (col,v) by spread
m.divides	col,v,rows,y,Sumr --> ls,rs,...	partition rows by (col,v)
m.tree	cols,rows,leaf,bins,y,Sumr --> tbl	greedy bi-tree on y
m.relevant	node,row --> rows	descend to leaf
m.vote	rows,klass,Klass --> v	klass-summary mid
Num.add,Sym.add (counts)	per-col tallies	mu/sd/mode/freq
Num.cut,Sym.cut	per-col candidates	split points

Self at near()[1] (dist 0). Skip self -> start at 2.

Apps reduce to primitives

app	reduction
ctree (exists)	tree + relevant + vote(Sym)
rtree (exists)	tree + relevant + near[1] at leaf
lnn classify	slice(near, 2, k+1) -> vote(Sym)
lnn regress	slice(near, 2, k+1) -> mean(Num)
lmeans seed	kmeans++: next centroid weighted by near(cents)[1] dist^2
lmeans assign	each row -> near(centroids)[1]
lnomaly	distx(row, near(rows)[k+1]) > quantile
lample smote	interpolate row with rand pick from slice(near,2,k+1)
lample diverse	greedy FFT: next = row maxing near(picked)[1] dist
lample centralize	lmeans, then 1 row per cluster
lample random	shuffle + slice
fastmap (anchors)	far twice = pick rand, far from it, far from that. far = near(rows)[floor(.9*#rows)]
layes	per-class Num/Sym counts; argmax P(k)*like
prototype reduce	keep row only if knn share its label

Files

file	role
lib.lua	generic helpers (list, stats, csv, cli, o)
lull.lua	AI primitives (Num, Sym, head, dist, tree)
ctree.lua	classification tree
rtree.lua	regression tree
lnn.lua	k-nearest classify/regress (stub)
lmeans.lua	kmeans cluster (stub)
layes.lua	naive bayes (stub)
lample.lua	smote/diverse/centralize/random (stub)
lnomaly.lua	outlier flag (stub)

Test strategy

Apps chain via Makefile:

• sample -> lnn/rtree, compare acc/RMSE
• proto-reduce -> lnn, compare acc with fewer rows
• anomaly -> labeled set, recall@k
• lnn vs layes vs ctree -> same data, compare via l.topTier

Makefile

# vim: ts=2 sw=2 sts=2 et :
SHELL := /bin/bash
OPEN  := $(shell command -v open 2>/dev/null || command -v xdg-open 2>/dev/null || echo true)
need   = @command -v $(1) >/dev/null || { printf "missing: %s (needed for %s)\n" $(1) $(2); exit 1; }

help: ## show help
	@gawk 'BEGIN {FS = ":.*?##"; \
	         printf "\nUsage:\n  make \033[36m<target>\033[0m [VAR=val ...]\n\ntargets:\n"} \
	       /^[~a-zA-Z0-9_%\.\/ -]+:.*?##/ { \
	         printf("  \033[36m%-20s\033[0m %s\n", $$1, $$2) | "sort" }' $(MAKEFILE_LIST)
	@printf "\ndefaults:\n"
	@gawk 'match($$0, /^([A-Za-z][A-Za-z0-9]*)[ \t]*\?=[ \t]*([^#]*[^# \t])[ \t]*#[ \t]*(.+)/, a) { \
	         printf("  \033[36m%-8s\033[0m = %-30s %s\n", a[1], a[2], a[3]) | "sort" }' $(MAKEFILE_LIST)

push: ## prompt msg, commit -am, push
	@read -p "Reason? " msg; git commit -am "$$msg"; git push; git status

define BASHRC
set -o vi
__gp(){ local b=$$(git branch --show-current 2>/dev/null); [[ -z $$b ]] && return
  [[ -n $$(git status --porcelain 2>/dev/null) ]] && b="$$b*"; echo " $$b"; }
__pw(){ pwd | awk -F/ '{print $$(NF-1)"/"$$NF}'; }
PS1='\[\e[36m\]$$(__pw)\[\e[33m\]$$(__gp) \[\e[0m\][\!]\$$ '
# ls colors (BSD via LSCOLORS, GNU via --color)
export CLICOLOR=1
export LSCOLORS=ExGxBxDxCxEgEdxbxgxcxd
export LS_COLORS='di=34:ln=35:so=32:pi=33:ex=31:bd=34;46:cd=34;43'
if command ls --color=auto / >/dev/null 2>&1; then
  alias ls='ls --color=auto'
else
  alias ls='ls -G'; fi
alias grep='grep --color=auto'
export LESS='-R'
alias ll='ls -la' gs='git status -s' gd='git diff' gl='git log --oneline -20'
# catppuccin: clone once per shell, wipe on exit; vi reuses it
D=$$(mktemp -d); trap 'rm -rf "$$D"' EXIT
( git clone -q --depth 1 $(Theme) "$$D/cat" 2>/dev/null & )
printf '%s\n' "$$VIMRC" > "$$D/vimrc"
alias vi='nvim --clean -u "$$D/vimrc" -c "set rtp^=$$D/cat" -c "colorscheme catppuccin-mocha"'
endef
export BASHRC

sh: ## launch tuned bash + catppuccin (cloned once, wiped on exit)
	$(call need,nvim,sh)
	$(call need,git,sh)
	@bash --rcfile <(echo "$$BASHRC") -i

Font   ?= 9         # for ~/tmp/%.pdf
Cols   ?= 2         # for ~/tmp/%.pdf
Orient ?= portrait  # for ~/tmp/%.pdf

doctor: ## check required tools (✓ found, ✗ missing)
	@for e in \
	   "bash|sh target, pdf trap (process subst)" \
	   "gawk|help target (self-doc)" \
	   "git|push target, sh prompt" \
	   "a2ps|pdf target (text → postscript)" \
	   "ps2pdf|pdf target (postscript → pdf, ghostscript)" \
	   "nvim|vi target (tuned editor)"; do \
	   c=$${e%%|*}; use=$${e##*|}; \
	   if command -v $$c >/dev/null; then \
	     printf "  \033[32m✓\033[0m %-10s used by: %s\n" "$$c" "$$use"; \
	   else \
	     printf "  \033[31m✗\033[0m %-10s missing — can't: %s\n" "$$c" "$$use"; fi; done
	@printf "\nmacOS: brew install gawk a2ps ghostscript neovim\n"
	@printf   "linux: apt install gawk a2ps ghostscript neovim\n"

define VIMRC
set nu rnu cursorline mouse=a termguicolors et ts=2 sw=2 sts=2 ai si
set ignorecase smartcase hlsearch incsearch scrolloff=8 signcolumn=yes
set splitbelow splitright wrap linebreak wildmenu clipboard=unnamedplus
syntax on | filetype plugin indent on
endef
export VIMRC

Theme ?= https://github.com/catppuccin/nvim # for vi

F ?= $(firstword $(wildcard *.md)) # for vi
vi: ## launch tuned nvim + catppuccin (wiped on exit)
	$(call need,nvim,vi)
	$(call need,git,vi)
	@D=$$(mktemp -d); trap "rm -rf $$D" EXIT; \
	 git clone -q --depth 1 $(Theme) $$D/cat; \
	 nvim --clean -c "$$VIMRC" \
	      -c "set rtp^=$$D/cat" -c "colorscheme catppuccin-mocha" $F

~/tmp/%.pdf : %.md Makefile ## md -> pdf via a2ps
	$(call need,a2ps,pdf)
	$(call need,ps2pdf,pdf)
	@mkdir -p ~/tmp
	@echo "pdfing : $@ ..."
	@a2ps -Bj --$(Orient) --line-numbers=1 --highlight-level=none \
	      --borders=no --right-footer="" --left-footer="" \
	      --footer="page %p." -M letter \
	      --font-size=$(Font) --columns $(Cols) \
	      -o - $< | ps2pdf - $@
	@$(OPEN) $@