Last active
June 23, 2024 15:34
-
-
Save mberman84/f092a28e4151dd5cecebfc58ac1cbc0e to your computer and use it in GitHub Desktop.
Steps to install Textgen WebUI
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # this tutorial assumes conda and git are both installed on your computer | |
| conda create -n tg python=3.10.9 | |
| conda activate tg | |
| pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 | |
| git clone https://github.com/oobabooga/text-generation-webui.git | |
| cd text-generation-webui | |
| pip install -r requirements.txt | |
| # GPU only: | |
| pip uninstall -y llama-cpp-python | |
| set CMAKE_ARGS="-DLLAMA_CUBLAS=on" | |
| set FORCE_CMAKE=1 | |
| pip install llama-cpp-python --no-cache-dir | |
| # If you get: ERROR: Failed building wheel for llama-cpp-python | |
| set "CMAKE_ARGS=-DLLAMA_OPENBLAS=on" | |
| set "FORCE_CMAKE=1" | |
| pip install llama-cpp-python --no-cache-dir | |
| # Put checker.py in your text-generation-webui folder | |
| python checker.py #Make sure you have cuda and it is enabled | |
| # If you get CUDA Setup failed despite GPU being available.: | |
| pip install bitsandbytes-windows | |
| # If you get AttributeError: module 'bitsandbytes.nn' has no attribute 'Linear4bit'. Did you mean: 'Linear8bitLt'? | |
| pip install git+https://github.com/huggingface/peft@27af2198225cbb9e049f548440f2bd0fba2204aa --force-reinstall --no-deps | |
| python server.py |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have worked for quite a while to solve an issue, to no avail::. I am working on Windows 11 machine with NVIDIA Geforce RTX 3060, 64 GB RAM, 2TB SSD with 24 cores. I have read over 50 articles, issues, blogs, videos, etc., and can't find the answer to the problem. After a lot of tweaks, I was down to the last line of the instruction -- spin up the server: python server.py. And Boom..... several errors in Gradio, including that gr.Box is deprecated. I have downgraded to lower versions of gradio (no luck, just different errors), have tried different versions of Python, Anaconda, GIT, etc., (as found in different blogs or on HuggingTree or GitHub, but I am still getting errors with gradio. So, i removed the virtual environment, deleted the text-generation-webui directory, and am back to scratch. Any suggestions on which tools are the best to use on Windows environment? Or should I take the advice of Bing Co-Pilot and Jeremy Morgan and install WSL to run LLMs locally? Thanks for your advice here.