Uche Ogbuji uogbuji

Word Loom is a convention for expressing language text and templates for AI language model-related uses, for example prompt templates. The format is based on TOML, and word looms are meant to be kept in resource directories for use with code invoking LLMs.

Basic principles:

Separation of code from natural language
- Must be a straightforward process to translate any natural language elements
Composability of natural language elements
Friendliness to mechanical comparisons (i.e. via diff)
Friendliness to traditional globalization (G11N) techniques

I've been using llama.cpp on Mac Silicon for months now, and my brother, Chimezie has been nudging me to give MLX a go. I finally set aside time today to get started, with an eventual goal of adding support for MLX model loading & usage in OgbujiPT. I've been warned it's rough around the edges, but it's been stimulating to play with. I thought I'd capture some of my notes, including some pitfalls I ran into, which might help anyone else trying to get into MLX in its current state.

As a quick bit of background I'll mention that MLX is very interesting because honestly, Apple has the most coherently engineered consumer and small-business-level hardware for AI workloads, with Apple Silicon and its unified memory. The news lately is all about Apple's AI fumbles, but I suspect their clever plan is to empower a community of developers to take the arrows in their back and build things out for them. The MLX

Python dev environment on Mac

User local install as much as possible, leaving system-wide kit alone, and no mega-distributions such as homebrew. Tested all the way through Ventura.

SSH

Make sure SSH is set up. Allow a remote computer to access your Mac and generate a new SSH Key. See appendix below.

There is automatically an ssh agent. Just use ssh-add. Note: if you use 1password, it has SSH agent management you can use.

Python dev environment on Linux

User local install as much as possible, leaving system-wide kit (e.g. python & python-pip .debs) alone. Tested on Debian, Ubuntu, Mint & ChromeOS/Crostini.

Environment

Set up the dev environment (this is the only system-wide part):

sudo apt install build-essential libsqlite3-dev sqlite3 bzip2 libbz2-dev zlib1g-dev libssl-dev openssl libgdbm-dev liblzma-dev libreadline-dev libncursesw5-dev libffi-dev uuid-dev

	#!/usr/bin/env python3
	'''
	update_constraints.py
	Helps manage Python package version constraints by updating based on currently installed versions.

	Features:

	- Caution - creates a temporary file rather than overwriting your original constraints
	- Preserves only packages that were in the original constraints file—doesn't add new ones
	- Error handling for packages that might be missing

	I'm on the board of Colorado Poets Center, and we're in fundraising season, with Colorado Gives Day coming up. [I'm hoping you can support the humanities in our state with a donation](https://www.coloradogives.org/donate/6d7hjf), or by spreading the word.

	$30 helps us add one new poet to the site. We now have 276 listed, adding 12-15 new poets each year.

	$50 helps us pay one poet to feature in our Tuesday night readings. 40 CPC poets were featured at the Boulder Bookstore or the R Gallery in 2023 & 2024.

	$50 helps us publish one electronic issue of the Colorado Poet newsletter.
	We've now published 10 quarterly issues, paying Kathy Winograd to interview 2 Colorado poets in each.

	$50 helps us pay one poet to read at community events.

	# Grr. Can't really use jupyter-as-script convention: https://twitter.com/uogbuji/status/1829325187965170154
	# %%
	'''
	test/llm_struct_probe.py
	'''
	import asyncio


	class llm_struct_probe:
	def __init__(self, model_path):

	# See https://github.com/uogbuji/OgbujiPT/issues/19
	import sys
	import asyncio

	import openai

	from ogbujipt.async_helper import schedule_callable # , openai_api_surrogate
	from ogbujipt import config
	from ogbujipt.prompting.basic import format
	from ogbujipt.prompting.model_style import ALPACA_DELIMITERS