Kyle Manna kylemanna

Finally used OpenAI's Deep Research for the first time

Building a Scalable AI Fine-Tuning Cluster with AMD Ryzen AI Max+ 395

This report outlines a model-independent framework for fine-tuning large AI models on a cluster of AMD Ryzen AI Max+ 395 nodes. The design supports a minimum of two nodes and scales to much larger deployments. We focus on optimizing fine-tuning efficiency using the XDNA 2 neural processing unit (NPU) in these chips, while keeping the setup accessible to developers of open-source AI models. Key areas include architecture and low-level optimizations, model splitting strategies, network and data throughput tuning, alternative computation models, and continuous benchmarking for improvements.

1. Architecture Analysis & Low-Level Optimization

XDNA 2 NPU vs CPU/GPU: AMD’s XDNA 2 NPU (built into Ryzen AI Max chips) is a specialized spatial dataflow engine optimized for AI workloads. It consists of a 2D array of compute tiles with a flexible interconnect and on-chip SRAM

TPM provides zero practical security

TPM (Trusted Platform Module) is as useful for preventing real attackers as the TSA is at preventing real terrorists. The architecture is fundamentally flawed and most existing implementations are completely broken. I thought this argument was settled decades ago[1] when "trusted computing" was introduced mostly as a way to provide DRM and ownership capabilities to organizations. It has largely failed to impact the consumer market when it was introduced back in the early 2000s. However, recently there seems to be a movement by certain parties to reintroduce this failed product back to the market. Microsoft argues that in order to use Windows 11, you need TPM 2.0 compatible hardware because[2]:

The Trusted Platform Module(TPM) requirement ena

Here is the best setup (I think so :D) for K-series Keychron keyboards on Linux.

Note: many newer Keychron keyboards use QMK as firmware and most tips here do not apply to them. Maybe the ones related to Bluetooth can be useful, but everything related to Apple's keyboard module (hid_apple) on Linux, won't work. As far as I know, all QMK-based boards use the hid_generic module instead. Examples of QMK-based boards are: Q, Q-Pro, V, K-Pro, etc.

Most of these commands have been tested on Ubuntu 20.04 and should also work on most Debian-based distributions. If a command happens not to work for you, take a look in the comment section.

Make Fn + F-keys work (NOT FOR QMK-BASED BOARDS)

Older Keychron keyboards (those not based on QMK) use the hid_apple driver on Linux, even in the Windows/Android mode, both in Bluetooth and Wired modes.

Python Flatbuffer Object -> JSON

Crude proof of concept, criticism welcome.

Usage

$ flatc --python monster.fbs
$ python3 ./python-flatbuffer-to-json.py
FIXME: Equipped is a table

{"color": 0, "equipped": null, "hp": 300, "inventory": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], "mana": 150, "name": "Orc", "path": [], "pos": {"x": 1.0, "y": 2.0, "z": 3.0}, "weapons": [{"damage": 3, "name": "Sword"}, {"damage": 5, "name": "Axe"}]}

Install NixOS on Oracle Cloud over Ubuntu 18.04 (make sure to use Ubuntu 18.04 or this may not work)

# install useful tools
sudo apt-get update
sudo apt-get install --no-install-recommends -y nano mc git

# prepare /boot
sudo umount /boot/efi
sudo mv /boot /boot.bak

	0810 b' from '
	0678 b' ssh2'
	00d8 b'%.48s:%.48s():%d (pid=%ld)\x00'
	0708 b'%s'
	0108 b'/usr/sbin/sshd\x00'
	0870 b'Accepted password for '
	01a0 b'Accepted publickey for '
	0c40 b'BN_bin2bn\x00'
	06d0 b'BN_bn2bin\x00'
	0958 b'BN_dup\x00'

	#!/usr/bin/python

	# Copyright 2009: dogbert <[email protected]>
	# Copyright 2023: 7enderhead (https://github.com/7enderhead/)
	#
	# This program is free software; you can redistribute it and/or modify
	# it under the terms of the GNU General Public License as published by
	# the Free Software Foundation; either version 2 of the License, or
	# (at your option) any later version.
	#

	[02/22/2025]

	Refs:
	1. https://mirrors.rit.edu/archlinux/iso/2025.02.01/archlinux-bootstrap-2025.02.01-x86_64.tar.zst
	2. https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/x86_64/alpine-virt-3.21.3-x86_64.iso
	3. https://wiki.alpinelinux.org/wiki/Replacing_non-Alpine_Linux_with_Alpine_remotely
	4. https://wiki.archlinux.org/index.php/installation_guide#Configure_the_system

	See also:
	1. Convert to Debian https://gist.github.com/zengxinhui/ee0ad6b7c7f99e2ead6cd0d2bd6641fd

	#!/usr/bin/env bash

	#export GIT_TRACE=1
	#export GIT_CURL_VERBOSE=1

	set -ex

	lfs_explode() {
	cd "$i"
	while true; do