Ishan Marikar ishan-marikar

Install a docker registry in a k3s kubernetes cluster

k3s doesn't have a builtin plugin for a local registry, so this is how to install the docker registry. The registry will be exposed via the default traefik ingress (on port 80 and 443).

Assumptions

resolvectl query registry.localhost returns 127.0.0.1 - on most Linux distros it does. If not, enable and configure systemd-resolved
K3s comes with traefik ingress controller by default, so the ingress for the registry service is configured for that.

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

4 Steps in Running LLaMA-7B on a M1 MacBook

The large language models usability

The problem with large language models is that you can’t run these locally on your laptop. Thanks to Georgi Gerganov and his llama.cpp project, it is now possible to run Meta’s LLaMA on a single computer without a dedicated GPU.

Running LLaMA

There are multiple steps involved in running LLaMA locally on a M1 Mac after downloading the model weights.

Using an old Barnes and Noble Nook Device

I got an ancient nook "simple touch" for free on a neighborhood Facebook group yesterday. Figured I could see if I at least like the form factor before getting myself a newer one.

Step 1: Charge it

When I got it, the battery was completely dead. I plugged it in to charge (thankfully a micro-USB charger). After about 20 minutes, it was ready to work on.

	# coding=utf-8
	# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software

	// Meant to work with the code in https://github.com/hrishioa/socrate, specifically the functions in the `gpt/base.ts`.
	// Created by Hrishi Olickel (@hrishioa) Sub 7 May

	import { Messages, askChatGPT } from '../base';

	async function generateRegexFromTypeSpec(typeSpec: string): Promise<string \| null> {
	//prettier-ignore
	const prompts = {
	systemPrompt: () => `You are a Typescript type to Regex converter that can only output valid Regexes.`,
	specPrompt: (spec: string) => `Convert the following TYPESCRIPT_TYPE into a valid Regex that matches the type.

	// Requires the gpt library from https://github.com/hrishioa/socrate and the progress bar library.
	// Created by Hrishi Olickel ([email protected]) (@hrishioa). Reach out if you have trouble running this.

	import { ThunkQueue } from '../../utils/simplethrottler';
	import {
	AcceptedModels,
	Messages,
	askChatGPT,
	getMessagesTokenCount,
	getProperJSONFromGPT,

	# Change these variables as necessary.
	main_package_path = ./cmd/example
	binary_name = example

	# ==================================================================================== #
	# HELPERS
	# ==================================================================================== #

	## help: print this help message
	.PHONY: help

	class ContactFinder {
	#db;
	#chatToFind;
	#dbName = "model-storage";
	#chatsCol = "chat";
	#contactCol = "contact";
	#groupCol = "participant";

	constructor(chatGroupName) {
	this.#chatToFind = chatGroupName;

	Mac: https://dl.appstreaming.autodesk.com/production/installers/Fusion%20360%20Client%20Downloader.dmg
	Windows: https://dl.appstreaming.autodesk.com/production/installers/Fusion%20360%20Client%20Downloader.exe