Skip to content

Instantly share code, notes, and snippets.

@siwonpawel

siwonpawel/LLAMA-2.md

Last active August 5, 2023 16:23
Show Gist options
  • Save siwonpawel/d0451f8642ced05bb036a2110025af60 to your computer and use it in GitHub Desktop.
Save siwonpawel/d0451f8642ced05bb036a2110025af60 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
LLAMA-2.md 
brew install miniconda
conda init zsh

# restart terminal

conda create -n mlc-chat-venv -c mlc-ai -c conda-forge mlc-chat-cli-nightly
conda activate mlc-chat-venv
mkdir -p dist/prebuilt
git clone https://github.com/mlc-ai/binary-mlc-llm-libs.git dist/prebuilt/lib

Instalation of models

cd dist/prebuilt
git clone https://huggingface.co/mlc-ai/mlc-chat-Llama-2-7b-chat-hf-q4f16_1

cd ../..
mlc_chat_cli --local-id llama-2-7b-chat-hf-q4f16_1

Uncensored

cd dist/prebuilt

git clone https://huggingface.co/mlc-ai/mlc-chat-llama2-7b-chat-uncensored-q4f16_1
cp lib/Llama-2-7b-chat-hf-q4f16_1-metal.so lib/llama2_7b_chat_uncensored-q4f16_1-metal.so

cd ../..
mlc_chat_cli --local-id llama-2-7b-chat-hf-q4f16_1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment