the-vampiire

I've been using llama.cpp on Mac Silicon for months now, and my brother, Chimezie has been nudging me to give MLX a go. I finally set aside time today to get started, with an eventual goal of adding support for MLX model loading & usage in OgbujiPT. I've been warned it's rough around the edges, but it's been stimulating to play with. I thought I'd capture some of my notes, including some pitfalls I ran into, which might help anyone else trying to get into MLX in its current state.

As a quick bit of background I'll mention that MLX is very interesting because honestly, Apple has the most coherently engineered consumer and small-business-level hardware for AI workloads, with Apple Silicon and its unified memory. The news lately is all about Apple's AI fumbles, but I suspect their clever plan is to empower a community of developers to take the arrows in their back and build things out for them. The MLX

Setup python, pip, virtualenv and virtualwrapper, with zsh on a new machine

Strongly inspired by https://gist.github.com/heymonkeyriot/9a2f429caff5c091d5429666fa080403.

Installing Python & Pip

On Ubuntu :

sudo apt install python3 python3-pip

Express & Postgres via Knex

Note: <example> is meant to denote text replaced by you (including brackets).

Setup

// global dependencies
npm install -g knex

	# /// script
	# dependencies = [ "transformers", "accelerate" ]
	# ///

	# run on 2xH200 rented from primeintellect.ai

	import gc

	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

	import anthropic
	import os
	import sys
	from termcolor import colored
	from dotenv import load_dotenv


	class ClaudeAgent:
	def __init__(self, api_key=None, model="claude-3-7-sonnet-20250219", max_tokens=4000):
	"""Initialize the Claude agent with API key and model."""

	#!/bin/bash

	# this forces Arena into full screen mode on startup, set back to 3 to reset
	# note that if you go into the Arena "Graphics" preference panel, it will reset all of these
	# and you will need to run these commands again
	defaults write com.wizards.mtga "Screenmanager Fullscreen mode" -integer 0
	defaults write com.wizards.mtga "Screenmanager Resolution Use Native" -integer 0

	# you can also replace the long complicated integer bit with any other scaled 16:9
	# resolution your system supports.

	0.0.0.0 feedback.microsoft-hohm.com
	0.0.0.0 search.msn.com
	0.0.0.0 a.ads1.msn.com
	0.0.0.0 a.ads2.msn.com
	0.0.0.0 a.rad.msn.com
	0.0.0.0 ac3.msn.com
	0.0.0.0 ads.msn.com
	0.0.0.0 ads1.msn.com
	0.0.0.0 b.ads1.msn.com
	0.0.0.0 b.rad.msn.com

	import fs from 'fs'
	import { makeExecutableSchema } from 'graphql-tools'
	import { graphql } from 'graphql'
	// the actual resolvers
	import resolvers from '../src/resolvers'
	// the mock service
	import mockMovieService from './mocks/mockMovieService'

	// a nice structure for test cases
	// found at https://hackernoon.com/extensive-graphql-testing-57e8760f1c25

	const { Model, AjvValidator } = require('objection');
	const pluralize = require('pluralize');

	class BaseModel extends Model {
	/**
	* Always use timestamps as default
	*/
	static get timestamps() {
	return true;
	}

	#include <stdio.h>
	#include <stdlib.h>
	#include <stdint.h>
	#ifdef _MSC_VER
	#include <intrin.h> /* for rdtscp and clflush */
	#pragma optimize("gt",on)
	#else
	#include <x86intrin.h> /* for rdtscp and clflush */
	#endif