Jinhui.Lin mintisan

Setup

On every machine in the cluster install openmpi and mlx-lm:

conda install conda-forge::openmpi
pip install -U mlx-lm

Next download the pipeline parallel run script. Download it to the same path on every machine:

NoteAfterNote-7
Reading and writing a USB drive connected to a Linux server using Termux, termux-usb, usbredirect, and QEMU on a smartphone that is not rooted
Published: May 19, 2024
Link: https://gist.github.com/NoteAfterNote/7a197233de3d60ff1e23ca90ed2f595a
Updated: May 29, 2024

Termux: Enable Wake-Lock

Smartphone Configuration

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

Ubuntu 21.04 VM with GPU acceleration under Hyper-V...?

Modern versions of Windows support GPU paravirtualization in Hyper-V with normal consumer graphics cards. This is used e.g. for graphics acceleration in Windows Sandbox, as well as WSLg. In some cases, it may be useful to create a normal VM with GPU acceleration using this feature, but this is not officially supported. People already figured out how to do it with Windows guests though, so why not do the same with Linux? It should be easy given that WSLg is open source and reasonably well documented, right?

Well... not quite. I managed to get it to run... but not well.

How to do it?

Verify driver support

	Title: Senior Engineer Task Execution Rule

	Applies to: All Tasks

	Rule:
	You are a senior engineer with deep experience building production-grade AI agents, automations, and workflow systems. Every task you execute must follow this procedure without exception:

	1.Clarify Scope First
	•Before writing any code, map out exactly how you will approach the task.
	•Confirm your interpretation of the objective.

	FROM qwen3:30b-a3b-q8_0

	TEMPLATE """{{- if .Messages }}
	{{- if or .System .Tools }}<\|im_start\|>system
	{{- if .System }}
	{{ .System }}
	{{- end }}
	{{- if .Tools }}

	# Tools

	// Updated: Aug. 20, 2024
	// Run: node testRegex.js whatever.txt
	// Live demo: https://jina.ai/tokenizer
	// LICENSE: Apache-2.0 (https://www.apache.org/licenses/LICENSE-2.0)
	// COPYRIGHT: Jina AI
	const fs = require('fs');
	const util = require('util');

	// Define variables for magic numbers
	const MAX_HEADING_LENGTH = 7;

	# coding=utf-8
	# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software

	# Prioritize NVIDIA packages
	wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
	sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

	# Fetch NVIDIA keys
	sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub


	# Add NVIDIA repos
	sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"

	#include <stdio.h>
	#include <string.h>
	#include <stdlib.h>
	#include "freertos/FreeRTOS.h"
	#include "freertos/task.h"
	#include "freertos/queue.h"
	#include "driver/gpio.h"
	#include "esp_pm.h"
	#include "esp_log.h"
	#include "esp_sleep.h"