Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggerganov/llama.cpp#5962
In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.
See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix
| import argparse | |
| import glob | |
| import os | |
| import torch | |
| from safetensors import safe_open | |
| from safetensors.torch import save_file | |
| ap = argparse.ArgumentParser() | |
| ap.add_argument("-m", "--model", help="Path to LLaVA v1.5 model") |
| !pip install transformers optimum | |
| !pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model_name_or_path = "TheBloke/zephyr-7B-beta-GPTQ" | |
| model = AutoModelForCausalLM.from_pretrained(model_name_or_path, | |
| device_map="auto", | |
| trust_remote_code=False, |
These are just my notes, and described process worked for me on my 1C robot
If anything goes wrong, having USB adapter for FEL flashing might be the only way to restore your robot
this is not an official guide
oh, and I'm not responsible for any damage blah blah
and huge thanks to Dennis and Hypfer, and everyone behind this root :)
to enter uboot shell on 1C you have to: 0. turn robot off normally
If you searched for this article I chances are you know what LKE, Traefik and Let's Encrypt are, else here is a quick primer:
"The Linode Kubernetes Engine (LKE) is a fully-managed container orchestration engine for deploying and managing containerized applications and workloads.". For more information please have a look at "Deploy and manage a cluster with Linode Kubernetes Engine"
| #!/bin/sh | |
| branch=$1 | |
| if [ ! -z "$1" ] | |
| then | |
| git branch -D $branch | |
| git push -d origin $branch | |
| else | |
| echo "Branch name is not specified" |
| # linux-meson.com/doku.php#howto | |
| make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig | |
| make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image dtbs | |
| mkimage -A arm64 -O linux -T kernel -C none -a 0x1080000 -e 0x1080000 -n linux-next -d arch/arm64/boot/Image ../uImage | |
| cp arch/arm64/boot/dts/amlogic/meson-gxbb-wetek-play2.dtb ../uImage BOOT_PARTITION | |
| # ROOTFS_PARTITION can be any arm64 Linux root filesystem |
| https://linuxconfig.org/how-to-install-the-nvidia-drivers-on-manjaro-18-linux | |
| https://github.com/dglt1/optimus-switch-gdm | |
| https://forum.manjaro.org/t/hdmi-monitor-not-working/94301/2 | |
| https://www.unixmen.com/install-oh-zsh-ubuntu-arch-linux-fedora/ | |
| https://starship.rs/guide/#%F0%9F%9A%80-installation | |
| https://gist.github.com/tomgco/d08fcf551b5e3157173d2f0701b50b46 | |
| https://wiki.archlinux.org/index.php/VirtualBox |
