Tested on NVIDIA RTX 4090, but these instructions also cover AMD and Mac in case you wanna try those.
This guide assumes you are running Linux (I ran this on Ubuntu).
Before you get excited:
# A simple script to demonstrate the sclicing and recombination of models at runtime | |
# inspired by mergekit | |
# Sadly, it doesn't work with quantisized models. | |
# | |
# public domain - silphendio | |
from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer | |
import torch | |
model_path = 'gpt2' # huggingface name or local folder |
import json | |
import os | |
import sys | |
from collections import defaultdict | |
from tqdm import tqdm | |
import argparse | |
import torch | |
from safetensors.torch import load_file, save_file |
The following installation methods were tried out.
With some trial and error, I managed to get a stand alone Ceph Object Storage Device (OSD) service working. The instructions to replicate the installation are as follows.
version: '3' | |
volumes: | |
postgres_data: | |
driver: local | |
services: | |
postgres: | |
image: postgres | |
volumes: |
This will setup a single node Consul cluster. Development mode is not used in order to allow data persistence. We'll be interacting with the Consul server from within the container for simplicity.
Create Consul configuration and persistent data directories
$ mkdir p consul/config consul/data
/system/busybox
and enable telnetd on your device (will add additional line into /system/autorun.sh
). Use it only on E5885, not on other device!telnet 192.168.8.1 2323
/system/busybox sh
// 브라우저에 GreaseMonkey(Firefox) TamperMonkey(Chrome) 을 설치하고, | |
// https://gist.github.com/foriequal0/154e73d3289d808e8ce94603f5eff6a4/raw/yaminjeongeum.user.js | |
// 위 링크를 브라우저에 입력하면 자동으로 스크립트를 설치하겠느냐는 창이 뜹니다. | |
// ==UserScript== | |
// @name yaminjeongeum | |
// @namespace yaminjeongeum.kr | |
// @description 야민정음 | |
// @include * | |
// @exclude file://* |