Skip to content

Instantly share code, notes, and snippets.

@mberman84

mberman84/gist:45545e48040ef6aafb6a1cb3442edb83

Created July 24, 2023 00:22
Show Gist options
  • Save mberman84/45545e48040ef6aafb6a1cb3442edb83 to your computer and use it in GitHub Desktop.
Save mberman84/45545e48040ef6aafb6a1cb3442edb83 to your computer and use it in GitHub Desktop.
Download ZIP
LLaMA 2 13b chat fp16 Install Instructions
Raw
gistfile1.txt
conda create -n textgen python=3.10.9
conda activate textgen
install pytorch: pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt
python server.py
# download model
# refresh model list
# load model
# switch to chat mode
@Theblabla1
Copy link

I found the solution. Issue was with prebuilds.

Change your requirements.txt file to this

aiofiles==23.1.0
fastapi==0.95.2
gradio_client==0.2.5
gradio==3.33.1

accelerate==0.21.0
colorama
datasets
einops
markdown
numpy
pandas
Pillow>=9.5.0
pyyaml
requests
safetensors==0.3.1
scipy
sentencepiece
tensorboard
tqdm
wandb
auto-gptq

llama-cpp-python


git+https://github.com/jllllll/GPTQ-for-LLaMa-CUDA.git
git+https://github.com/huggingface/peft@96c0277a1b9a381b10ab34dbf84917f9b3b992e6
git+https://github.com/huggingface/transformers@baf1daa58eb2960248fd9f7c3af0ed245b8ce4af

git+https://github.com/jllllll/exllama

bitsandbytes==0.41.1; platform_system != "Windows"
https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl; platform_system == "Windows"


# ctransformers
https://github.com/jllllll/ctransformers-cuBLAS-wheels/releases/download/AVX2/ctransformers-0.2.20+cu117-py3-none-any.whl

Additional requirements

cc: @chiefdataofficer

I dont understand which requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment