Jioh L. Jung ziozzang

Running GitHub Copilot VSCode extension against local Code Llama model

Tested on NVIDIA RTX 4090, but these instructions also cover AMD and Mac in case you wanna try those.
This guide assumes you are running Linux (I ran this on Ubuntu).

Before you get excited:

The following installation methods were tried out.

Working instances of Ceph OSD service

Vagrant-based solution

Container-based solution (partially working)

With some trial and error, I managed to get a stand alone Ceph Object Storage Device (OSD) service working. The instructions to replicate the installation are as follows.

set http or socks proxy environment variables

# set http proxy
export http_proxy=http://PROXYHOST:PROXYPORT

# set http proxy with user and password
export http_proxy=http://USERNAME:PASSWORD@PROXYHOST:PROXYPORT

# set http proxy with user and password (with special characters)

Setup Consul with ACL

This will setup a single node Consul cluster. Development mode is not used in order to allow data persistence. We'll be interacting with the Consul server from within the container for simplicity.

Create Consul configuration and persistent data directories

$ mkdir p consul/config consul/data

CFSSL as an external CA for non-ha kubeadm intialized clusters

Using cfssl to Create an External CA Infrastructure

Install cfssl

# This requires an existing Go environment with GOPATH set
go get -u github.com/cloudflare/cfssl/cmd/...

Huawei E5885 current firmware backup manual.

Download modified usbloader, which will copy proper busybox into /system/busybox and enable telnetd on your device (will add additional line into /system/autorun.sh). Use it only on E5885, not on other device!
Load it using balong-usbdload. Refer to this disassembling manual.
After loading, wait about one minute, disconnect device from the computer and power off the device holding power button for ~15 seconds.
Insert MicroSD card to the device and power it on.
telnet 192.168.8.1 2323

/system/busybox sh

	#!/bin/bash
	set -euo pipefail
	trap 'echo "at line $LINENO, exit code $? from $BASH_COMMAND" >&2; exit 1' ERR

	# This is a Claude Code hook to stop it saying "you are right".
	#
	# Installation:
	# 1. Save this script and chmod +x it to make it executable.
	# 2. Within Claude Code, /hooks / UserPromptSubmit > Add a new hook (this file)
	#

	# A simple script to demonstrate the sclicing and recombination of models at runtime
	# inspired by mergekit
	# Sadly, it doesn't work with quantisized models.
	#
	# public domain - silphendio

	from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
	import torch

	model_path = 'gpt2' # huggingface name or local folder

	import json
	import os
	import sys
	from collections import defaultdict
	from tqdm import tqdm
	import argparse
	import torch

	from safetensors.torch import load_file, save_file

	version: '3'

	volumes:
	postgres_data:
	driver: local

	services:
	postgres:
	image: postgres
	volumes: