-
-
Save mberman84/9e008131d96af27256cc9cb53ad834cf to your computer and use it in GitHub Desktop.
| # Make sure you have Anaconda installed | |
| # This tutorial assumes you have an Nvidia GPU, but you can find the non-GPU version on the Textgen WebUI github | |
| # More information found here: https://github.com/oobabooga/text-generation-webui | |
| conda create -n textgen python=3.10.9 | |
| conda activate textgen | |
| pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 | |
| git clone https://github.com/oobabooga/text-generation-webui | |
| cd text-generation-webui | |
| python -m pip install -r requirements.txt | |
| python -m torch.utils.collect_env # optional, checks that you have CUDA enabled | |
| # if you have trouble with CUDA being enabled in torch, try this: | |
| conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia | |
| # if you run into the chardet and cchardet issues I did, try this: | |
| python -m pip install chardet | |
| python -m pip install cchardet | |
| python server.py |
(textgeneration) C:\Users\Patrick>python server.py python: can't open file 'C:\Users\Patrick\server.py': [Errno 2] No such file or directory
this was the cause of the error. conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
please help what is this issue
PS C:\Users\Hp\cChardet> pip install twint
Defaulting to user installation because normal site-packages is not writeable
Collecting twint
Using cached twint-2.1.20-py3-none-any.whl
Requirement already satisfied: aiohttp in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from twint) (3.9.3)
Collecting aiodns (from twint)
Using cached aiodns-3.1.1-py3-none-any.whl.metadata (4.0 kB)
Requirement already satisfied: beautifulsoup4 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from twint) (4.12.3)
Collecting cchardet (from twint)
Using cached cchardet-2.1.7.tar.gz (653 kB)
Preparing metadata (setup.py) ... done
Collecting elasticsearch (from twint)
Using cached elasticsearch-8.12.1-py3-none-any.whl.metadata (5.3 kB)
Requirement already satisfied: pysocks in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from twint) (1.7.1)
Requirement already satisfied: pandas in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from twint) (2.2.1)
Collecting aiohttp-socks (from twint)
Using cached aiohttp_socks-0.8.4-py3-none-any.whl.metadata (3.1 kB)
Collecting schedule (from twint)
Using cached schedule-1.2.1-py2.py3-none-any.whl.metadata (3.3 kB)
Collecting geopy (from twint)
Using cached geopy-2.4.1-py3-none-any.whl.metadata (6.8 kB)
Collecting fake-useragent (from twint)
Using cached fake_useragent-1.4.0-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: googletransx in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from twint) (2.4.2)
Collecting pycares>=4.0.0 (from aiodns->twint)
Using cached pycares-4.4.0-cp312-cp312-win_amd64.whl.metadata (4.5 kB)
Requirement already satisfied: aiosignal>=1.1.2 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from aiohttp->twint) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from aiohttp->twint) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from aiohttp->twint) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from aiohttp->twint) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from aiohttp->twint) (1.9.4)
Collecting python-socks<3.0.0,>=2.4.3 (from python-socks[asyncio]<3.0.0,>=2.4.3->aiohttp-socks->twint)
Using cached python_socks-2.4.4-py3-none-any.whl.metadata (7.1 kB)
Requirement already satisfied: soupsieve>1.2 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from beautifulsoup4->twint) (2.5)
Collecting elastic-transport<9,>=8 (from elasticsearch->twint)
Using cached elastic_transport-8.12.0-py3-none-any.whl.metadata (3.5 kB)
Collecting geographiclib<3,>=1.52 (from geopy->twint)
Using cached geographiclib-2.0-py3-none-any.whl.metadata (1.4 kB)
Requirement already satisfied: requests in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from googletransx->twint) (2.31.0)
Requirement already satisfied: numpy<2,>=1.26.0 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from pandas->twint) (1.26.4)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from pandas->twint) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from pandas->twint) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from pandas->twint) (2024.1)
Requirement already satisfied: urllib3<3,>=1.26.2 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from elastic-transport<9,>=8->elasticsearch->twint) (2.2.1)
Requirement already satisfied: certifi in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from elastic-transport<9,>=8->elasticsearch->twint) (2024.2.2)
Collecting cffi>=1.5.0 (from pycares>=4.0.0->aiodns->twint)
Using cached cffi-1.16.0-cp312-cp312-win_amd64.whl.metadata (1.5 kB)
Requirement already satisfied: six>=1.5 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from python-dateutil>=2.8.2->pandas->twint) (1.16.0)
Collecting async-timeout>=3.0.1 (from python-socks[asyncio]<3.0.0,>=2.4.3->aiohttp-socks->twint)
Using cached async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)
Requirement already satisfied: idna>=2.0 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from yarl<2.0,>=1.0->aiohttp->twint) (3.6)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\hp\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from requests->googletransx->twint) (3.3.2)
Collecting pycparser (from cffi>=1.5.0->pycares>=4.0.0->aiodns->twint)
Using cached pycparser-2.21-py2.py3-none-any.whl.metadata (1.1 kB)
Using cached aiodns-3.1.1-py3-none-any.whl (5.4 kB)
Using cached aiohttp_socks-0.8.4-py3-none-any.whl (9.6 kB)
Using cached elasticsearch-8.12.1-py3-none-any.whl (432 kB)
Using cached fake_useragent-1.4.0-py3-none-any.whl (15 kB)
Using cached geopy-2.4.1-py3-none-any.whl (125 kB)
Using cached schedule-1.2.1-py2.py3-none-any.whl (11 kB)
Using cached elastic_transport-8.12.0-py3-none-any.whl (59 kB)
Using cached geographiclib-2.0-py3-none-any.whl (40 kB)
Using cached pycares-4.4.0-cp312-cp312-win_amd64.whl (76 kB)
Using cached python_socks-2.4.4-py3-none-any.whl (52 kB)
Using cached async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Using cached cffi-1.16.0-cp312-cp312-win_amd64.whl (181 kB)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Building wheels for collected packages: cchardet
Building wheel for cchardet (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [11 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-312
creating build\lib.win-amd64-cpython-312\cchardet
copying src\cchardet\version.py -> build\lib.win-amd64-cpython-312\cchardet
copying src\cchardet_init_.py -> build\lib.win-amd64-cpython-312\cchardet
running build_ext
building 'cchardet._cchardet' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cchardet
Running setup.py clean for cchardet
Failed to build cchardet
ERROR: Could not build wheels for cchardet, which is required to install pyproject.toml-based projects
PS C:\Users\Hp\cChardet>
22:29:20-139114 ERROR Failed to load the model.
Traceback (most recent call last):
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\utils\import_utils.py", line 1364, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\Users\onego\anaconda3\envs\textgen\lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\models\llama\modeling_llama.py", line 55, in
from flash_attn import flash_attn_func, flash_attn_varlen_func
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\flash_attn_init.py", line 3, in
from flash_attn.flash_attn_interface import (
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\flash_attn\flash_attn_interface.py", line 8, in
import flash_attn_2_cuda as flash_attn_cuda
ImportError: DLL load failed while importing flash_attn_2_cuda: The specified procedure could not be found.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Windows\System32\text-generation-webui\modules\ui_model_menu.py", line 220, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
File "C:\Windows\System32\text-generation-webui\modules\models.py", line 87, in load_model
output = load_func_maploader
File "C:\Windows\System32\text-generation-webui\modules\models.py", line 235, in huggingface_loader
model = LoaderClass.from_pretrained(path_to_model, **params)
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\models\auto\auto_factory.py", line 565, in from_pretrained
model_class = _get_model_class(config, cls._model_mapping)
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\models\auto\auto_factory.py", line 387, in _get_model_class
supported_models = model_mapping[type(config)]
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\models\auto\auto_factory.py", line 740, in getitem
return self._load_attr_from_module(model_type, model_name)
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\models\auto\auto_factory.py", line 754, in _load_attr_from_module
return getattribute_from_module(self._modules[module_name], attr)
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\models\auto\auto_factory.py", line 698, in getattribute_from_module
if hasattr(module, attr):
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\utils\import_utils.py", line 1354, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\onego\anaconda3\envs\textgen\lib\site-packages\transformers\utils\import_utils.py", line 1366, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
DLL load failed while importing flash_attn_2_cuda: The specified procedure could not be found.