frandi/install-text-generation-web-ui.md

Last active September 30, 2023 14:38

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/frandi/7f931965617de927a50108fb7d02ffb4.js"></script>
Save frandi/7f931965617de927a50108fb7d02ffb4 to your computer and use it in GitHub Desktop.

Download ZIP

install text generation web ui

Raw

install-text-generation-web-ui.md

Install CUDA

https://developer.nvidia.com/cuda-downloads

set env var (.profile)

export PATH=/usr/local/cuda-12.2/compat:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.2/compat:$LD_LIBRARY_PATH

install miniconda

https://docs.conda.io/projects/miniconda/en/latest/miniconda-install.html

install pytorch

https://pytorch.org/get-started/locally/ or

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

optional, checks that you have CUDA enabled

python -m torch.utils.collect_env

clone Text Generation Web UI

git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
python -m pip install -r requirements.txt

start web ui

python server.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment