Skip to content

Instantly share code, notes, and snippets.

@machinelearnear

machinelearnear/freewilly-stablebeluga-on-google-colab-w-text-generation-webui.ipynb

Created July 28, 2023 14:10
Show Gist options
  • Save machinelearnear/2e8bcac4684a9181cad77deae7f765c1 to your computer and use it in GitHub Desktop.
Save machinelearnear/2e8bcac4684a9181cad77deae7f765c1 to your computer and use it in GitHub Desktop.
Download ZIP
FreeWilly-StableBeluga on Google Colab w/ text-generation-webui
Display the source blob
Display the rendered blob
Raw
freewilly-stablebeluga-on-google-colab-w-text-generation-webui.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/machinelearnear/2e8bcac4684a9181cad77deae7f765c1/freewilly-stablebeluga-on-google-colab-w-text-generation-webui.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VGbfhIjnaNK4"
},
"source": [
"# Run `StableBeluga` with `text-generation-webui` on Google Colab\n",
"- https://huggingface.co/TheBloke/StableBeluga2-GPTQ\n",
"- https://huggingface.co/stabilityai/StableBeluga-7B\n",
"- https://github.com/oobabooga/text-generation-webui"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "gogfEEqiafRR",
"outputId": "c3a375d1-c1fd-409d-e0ba-4636f17b30bb"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Cloning into 'text-generation-webui'...\n",
"remote: Enumerating objects: 9741, done.\u001b[K\n",
"remote: Counting objects: 100% (521/521), done.\u001b[K\n",
"remote: Compressing objects: 100% (277/277), done.\u001b[K\n",
"remote: Total 9741 (delta 328), reused 382 (delta 239), pack-reused 9220\u001b[K\n",
"Receiving objects: 100% (9741/9741), 3.29 MiB | 1.74 MiB/s, done.\n",
"Resolving deltas: 100% (6505/6505), done.\n",
"/content/text-generation-webui\n",
"Collecting git+https://github.com/huggingface/peft@96c0277a1b9a381b10ab34dbf84917f9b3b992e6 (from -r requirements.txt (line 21))\n",
" Cloning https://github.com/huggingface/peft (to revision 96c0277a1b9a381b10ab34dbf84917f9b3b992e6) to /tmp/pip-req-build-d1xflvle\n",
" Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft /tmp/pip-req-build-d1xflvle\n",
" Running command git rev-parse -q --verify 'sha^96c0277a1b9a381b10ab34dbf84917f9b3b992e6'\n",
" Running command git fetch -q https://github.com/huggingface/peft 96c0277a1b9a381b10ab34dbf84917f9b3b992e6\n",
" Running command git checkout -q 96c0277a1b9a381b10ab34dbf84917f9b3b992e6\n",
" Resolved https://github.com/huggingface/peft to commit 96c0277a1b9a381b10ab34dbf84917f9b3b992e6\n",
" Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
" Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
" Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
"Ignoring bitsandbytes: markers 'platform_system == \"Windows\"' don't match your environment\n",
"Ignoring auto-gptq: markers 'platform_system == \"Windows\"' don't match your environment\n",
"Collecting auto-gptq==0.3.0+cu117 (from -r requirements.txt (line 25))\n",
" Downloading https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.3.0/auto_gptq-0.3.0+cu117-cp310-cp310-linux_x86_64.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m12.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hIgnoring exllama: markers 'platform_system == \"Windows\"' don't match your environment\n",
"Collecting exllama==0.0.8+cu117 (from -r requirements.txt (line 27))\n",
" Downloading https://github.com/jllllll/exllama/releases/download/0.0.8/exllama-0.0.8+cu117-cp310-cp310-linux_x86_64.whl (354 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m354.3/354.3 kB\u001b[0m \u001b[31m17.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hIgnoring llama-cpp-python: markers 'platform_system == \"Windows\"' don't match your environment\n",
"Ignoring llama-cpp-python-cuda: markers 'platform_system == \"Windows\"' don't match your environment\n",
"Collecting llama-cpp-python-cuda==0.1.77+cu117 (from -r requirements.txt (line 33))\n",
" Downloading https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.1.77+cu117-cp310-cp310-linux_x86_64.whl (2.9 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.9/2.9 MB\u001b[0m \u001b[31m33.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting accelerate==0.21.0 (from -r requirements.txt (line 1))\n",
" Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting colorama (from -r requirements.txt (line 2))\n",
" Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n",
"Collecting datasets (from -r requirements.txt (line 3))\n",
" Downloading datasets-2.14.1-py3-none-any.whl (492 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m492.4/492.4 kB\u001b[0m \u001b[31m13.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting einops (from -r requirements.txt (line 4))\n",
" Downloading einops-0.6.1-py3-none-any.whl (42 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.2/42.2 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting fastapi==0.95.2 (from -r requirements.txt (line 5))\n",
" Downloading fastapi-0.95.2-py3-none-any.whl (56 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.0/57.0 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting gradio_client==0.2.5 (from -r requirements.txt (line 6))\n",
" Downloading gradio_client-0.2.5-py3-none-any.whl (288 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.1/288.1 kB\u001b[0m \u001b[31m12.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting gradio==3.33.1 (from -r requirements.txt (line 7))\n",
" Downloading gradio-3.33.1-py3-none-any.whl (20.0 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m72.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: markdown in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (3.4.4)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (1.22.4)\n",
"Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 10)) (1.5.3)\n",
"Collecting Pillow>=9.5.0 (from -r requirements.txt (line 11))\n",
" Downloading Pillow-10.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.4 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m105.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (6.0.1)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (2.27.1)\n",
"Collecting safetensors==0.3.1 (from -r requirements.txt (line 14))\n",
" Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m77.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (1.10.1)\n",
"Collecting sentencepiece (from -r requirements.txt (line 16))\n",
" Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m85.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 17)) (2.12.3)\n",
"Collecting transformers==4.31.* (from -r requirements.txt (line 18))\n",
" Downloading transformers-4.31.0-py3-none-any.whl (7.4 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.4/7.4 MB\u001b[0m \u001b[31m118.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 19)) (4.65.0)\n",
"Collecting wandb (from -r requirements.txt (line 20))\n",
" Downloading wandb-0.15.7-py3-none-any.whl (2.1 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m96.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting bitsandbytes==0.41.0 (from -r requirements.txt (line 22))\n",
" Downloading bitsandbytes-0.41.0-py3-none-any.whl (92.6 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.6/92.6 MB\u001b[0m \u001b[31m11.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting llama-cpp-python==0.1.77 (from -r requirements.txt (line 29))\n",
" Downloading llama_cpp_python-0.1.77.tar.gz (1.6 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m91.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
" Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
" Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
"Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 1)) (23.1)\n",
"Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 1)) (5.9.5)\n",
"Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 1)) (2.0.1+cu118)\n",
"Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /usr/local/lib/python3.10/dist-packages (from fastapi==0.95.2->-r requirements.txt (line 5)) (1.10.12)\n",
"Collecting starlette<0.28.0,>=0.27.0 (from fastapi==0.95.2->-r requirements.txt (line 5))\n",
" Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 6)) (2023.6.0)\n",
"Collecting httpx (from gradio_client==0.2.5->-r requirements.txt (line 6))\n",
" Downloading httpx-0.24.1-py3-none-any.whl (75 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.4/75.4 kB\u001b[0m \u001b[31m9.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting huggingface-hub>=0.13.0 (from gradio_client==0.2.5->-r requirements.txt (line 6))\n",
" Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m29.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 6)) (4.7.1)\n",
"Collecting websockets (from gradio_client==0.2.5->-r requirements.txt (line 6))\n",
" Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m17.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting aiofiles (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)\n",
"Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.8.5)\n",
"Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (4.2.2)\n",
"Collecting ffmpy (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading ffmpy-0.3.1.tar.gz (5.5 kB)\n",
" Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.1.2)\n",
"Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.0.0)\n",
"Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (2.1.3)\n",
"Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.7.1)\n",
"Collecting mdit-py-plugins<=0.3.3 (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting orjson (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading orjson-3.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m17.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting pydub (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
"Requirement already satisfied: pygments>=2.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (2.14.0)\n",
"Collecting python-multipart (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m5.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting semantic-version (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
"Collecting uvicorn>=0.14.0 (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading uvicorn-0.23.1-py3-none-any.whl (59 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.5/59.5 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.*->-r requirements.txt (line 18)) (3.12.2)\n",
"Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.*->-r requirements.txt (line 18)) (2022.10.31)\n",
"Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.31.*->-r requirements.txt (line 18))\n",
" Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m124.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting diskcache>=5.6.1 (from llama-cpp-python==0.1.77->-r requirements.txt (line 29))\n",
" Downloading diskcache-5.6.1-py3-none-any.whl (45 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.6/45.6 kB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->-r requirements.txt (line 3)) (9.0.0)\n",
"Collecting dill<0.3.8,>=0.3.0 (from datasets->-r requirements.txt (line 3))\n",
" Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m15.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting xxhash (from datasets->-r requirements.txt (line 3))\n",
" Downloading xxhash-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.5/212.5 kB\u001b[0m \u001b[31m22.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting multiprocess (from datasets->-r requirements.txt (line 3))\n",
" Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m17.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 10)) (2.8.2)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 10)) (2022.7.1)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (1.26.16)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (2023.7.22)\n",
"Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (2.0.12)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (3.4)\n",
"Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.4.0)\n",
"Requirement already satisfied: grpcio>=1.48.2 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.56.2)\n",
"Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (2.17.3)\n",
"Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.0.0)\n",
"Requirement already satisfied: protobuf>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (3.20.3)\n",
"Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (67.7.2)\n",
"Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (0.7.1)\n",
"Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (2.3.6)\n",
"Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (0.41.0)\n",
"Requirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 20)) (8.1.6)\n",
"Collecting GitPython!=3.1.29,>=1.0.0 (from wandb->-r requirements.txt (line 20))\n",
" Downloading GitPython-3.1.32-py3-none-any.whl (188 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m188.5/188.5 kB\u001b[0m \u001b[31m21.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting sentry-sdk>=1.0.0 (from wandb->-r requirements.txt (line 20))\n",
" Downloading sentry_sdk-1.28.1-py2.py3-none-any.whl (214 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m214.7/214.7 kB\u001b[0m \u001b[31m23.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting docker-pycreds>=0.4.0 (from wandb->-r requirements.txt (line 20))\n",
" Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n",
"Collecting pathtools (from wandb->-r requirements.txt (line 20))\n",
" Downloading pathtools-0.1.2.tar.gz (11 kB)\n",
" Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Collecting setproctitle (from wandb->-r requirements.txt (line 20))\n",
" Downloading setproctitle-1.3.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)\n",
"Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 20)) (1.4.4)\n",
"Collecting rouge (from auto-gptq==0.3.0+cu117->-r requirements.txt (line 25))\n",
" Downloading rouge-1.0.1-py3-none-any.whl (13 kB)\n",
"Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.4)\n",
"Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (4.3.3)\n",
"Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.12.0)\n",
"Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from docker-pycreds>=0.4.0->wandb->-r requirements.txt (line 20)) (1.16.0)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (23.1.0)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (6.0.4)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (4.0.2)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (1.9.2)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (1.4.0)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (1.3.1)\n",
"Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 20))\n",
" Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (5.3.1)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (0.3.0)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (4.9)\n",
"Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard->-r requirements.txt (line 17)) (1.3.1)\n",
"Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.1.2)\n",
"Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.33.1->-r requirements.txt (line 7)) (2.0.2)\n",
"INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
"Collecting mdit-py-plugins<=0.3.3 (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)\n",
" Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)\n",
"INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
" Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)\n",
" Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)\n",
" Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)\n",
" Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)\n",
" Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)\n",
"INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
" Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)\n",
"Collecting markdown-it-py[linkify]>=2.0.0 (from gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m10.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m10.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.28.0,>=0.27.0->fastapi==0.95.2->-r requirements.txt (line 5)) (3.7.1)\n",
"Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (1.11.1)\n",
"Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (3.1)\n",
"Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (2.0.0)\n",
"Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (3.25.2)\n",
"Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (16.0.6)\n",
"Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio==3.33.1->-r requirements.txt (line 7))\n",
" Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m7.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting httpcore<0.18.0,>=0.15.0 (from httpx->gradio_client==0.2.5->-r requirements.txt (line 6))\n",
" Downloading httpcore-0.17.3-py3-none-any.whl (74 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.5/74.5 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio_client==0.2.5->-r requirements.txt (line 6)) (1.3.0)\n",
"Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (1.1.0)\n",
"Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (0.11.0)\n",
"Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (4.41.1)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (1.4.4)\n",
"Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (3.1.0)\n",
"Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.28.0,>=0.27.0->fastapi==0.95.2->-r requirements.txt (line 5)) (1.1.2)\n",
"Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 20))\n",
" Downloading smmap-5.0.0-py3-none-any.whl (24 kB)\n",
"Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.19.3)\n",
"Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio==3.33.1->-r requirements.txt (line 7)) (1.0.2)\n",
"Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (0.5.0)\n",
"Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard->-r requirements.txt (line 17)) (3.2.2)\n",
"Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (1.3.0)\n",
"Building wheels for collected packages: llama-cpp-python, peft, ffmpy, pathtools\n",
" Building wheel for llama-cpp-python (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.77-cp310-cp310-linux_x86_64.whl size=275941 sha256=be66183afa23ac8009be0fda7fe522ebb47aeb4394865192771018d6e0f52f3a\n",
" Stored in directory: /root/.cache/pip/wheels/aa/ed/39/87f2ad350dbbf13b600ac744899186b8647c5323c62e2bb348\n",
" Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for peft: filename=peft-0.5.0.dev0-py3-none-any.whl size=73132 sha256=5ec96bb1522a2be92dec4919f03dd63305630edf7037cb494279a82f7c9489c9\n",
" Stored in directory: /root/.cache/pip/wheels/ff/57/c1/a023c490307cd8ffa3b61c86c48d9767f0bb850053af18674b\n",
" Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=f37d77864eaa1690937d1cb3ec3145a50606d01d8f354d9b51143308173191d6\n",
" Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf\n",
" Building wheel for pathtools (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8791 sha256=746ec9e2b53417a749b5b54b0d3251fcef732fd9f1cb37de2c002d6317981192\n",
" Stored in directory: /root/.cache/pip/wheels/e7/f3/22/152153d6eb222ee7a56ff8617d80ee5207207a8c00a7aab794\n",
"Successfully built llama-cpp-python peft ffmpy pathtools\n",
"Installing collected packages: tokenizers, sentencepiece, safetensors, pydub, pathtools, ffmpy, bitsandbytes, xxhash, websockets, smmap, setproctitle, sentry-sdk, semantic-version, rouge, python-multipart, Pillow, orjson, markdown-it-py, h11, einops, docker-pycreds, diskcache, dill, colorama, aiofiles, uvicorn, starlette, multiprocess, mdit-py-plugins, llama-cpp-python-cuda, llama-cpp-python, huggingface-hub, httpcore, gitdb, transformers, httpx, GitPython, fastapi, wandb, gradio_client, datasets, gradio, accelerate, peft, exllama, auto-gptq\n",
" Attempting uninstall: Pillow\n",
" Found existing installation: Pillow 9.4.0\n",
" Uninstalling Pillow-9.4.0:\n",
" Successfully uninstalled Pillow-9.4.0\n",
" Attempting uninstall: markdown-it-py\n",
" Found existing installation: markdown-it-py 3.0.0\n",
" Uninstalling markdown-it-py-3.0.0:\n",
" Successfully uninstalled markdown-it-py-3.0.0\n",
" Attempting uninstall: mdit-py-plugins\n",
" Found existing installation: mdit-py-plugins 0.4.0\n",
" Uninstalling mdit-py-plugins-0.4.0:\n",
" Successfully uninstalled mdit-py-plugins-0.4.0\n",
"Successfully installed GitPython-3.1.32 Pillow-10.0.0 accelerate-0.21.0 aiofiles-23.1.0 auto-gptq-0.3.0+cu117 bitsandbytes-0.41.0 colorama-0.4.6 datasets-2.14.1 dill-0.3.7 diskcache-5.6.1 docker-pycreds-0.4.0 einops-0.6.1 exllama-0.0.8+cu117 fastapi-0.95.2 ffmpy-0.3.1 gitdb-4.0.10 gradio-3.33.1 gradio_client-0.2.5 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 huggingface-hub-0.16.4 llama-cpp-python-0.1.77 llama-cpp-python-cuda-0.1.77+cu117 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 multiprocess-0.70.15 orjson-3.9.2 pathtools-0.1.2 peft-0.5.0.dev0 pydub-0.25.1 python-multipart-0.0.6 rouge-1.0.1 safetensors-0.3.1 semantic-version-2.10.0 sentencepiece-0.1.99 sentry-sdk-1.28.1 setproctitle-1.3.2 smmap-5.0.0 starlette-0.27.0 tokenizers-0.13.3 transformers-4.31.0 uvicorn-0.23.1 wandb-0.15.7 websockets-11.0.3 xxhash-3.2.0\n"
]
},
{
"output_type": "display_data",
"data": {
"application/vnd.colab-display-data+json": {
"pip_warning": {
"packages": [
"PIL"
]
}
}
},
"metadata": {}
}
],
"source": [
"!git clone https://github.com/oobabooga/text-generation-webui\n",
"%cd text-generation-webui\n",
"!pip install -r requirements.txt"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"id": "JBx8MycbamH-"
},
"outputs": [],
"source": [
"!pip install bitsandbytes>=0.39.0"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_hB0IY7Tc--Q",
"outputId": "76c9ac8b-2d64-4bb0-88c8-6e5fb545d268"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Downloading the model to models/TheBloke_StableBeluga2-GPTQ\n",
"100% 7.02k/7.02k [00:00<00:00, 43.2MiB/s]\n",
"100% 15.6k/15.6k [00:00<00:00, 88.6MiB/s]\n",
"100% 4.77k/4.77k [00:00<00:00, 33.2MiB/s]\n",
"100% 679/679 [00:00<00:00, 6.52MiB/s]\n",
"100% 137/137 [00:00<00:00, 1.13MiB/s]\n",
"100% 35.3G/35.3G [02:53<00:00, 203MiB/s]\n",
"100% 183/183 [00:00<00:00, 1.51MiB/s]\n",
"100% 411/411 [00:00<00:00, 3.44MiB/s]\n",
"100% 1.84M/1.84M [00:00<00:00, 4.52MiB/s]\n",
"100% 500k/500k [00:00<00:00, 295MiB/s]\n",
"100% 649/649 [00:00<00:00, 5.10MiB/s]\n",
"Downloading the model to models/stabilityai_StableBeluga-7B\n",
"100% 5.18k/5.18k [00:00<00:00, 34.5MiB/s]\n",
"100% 7.02k/7.02k [00:00<00:00, 29.0MiB/s]\n",
"100% 5.18k/5.18k [00:00<00:00, 27.3MiB/s]\n",
"100% 4.77k/4.77k [00:00<00:00, 28.3MiB/s]\n",
"100% 583/583 [00:00<00:00, 4.26MiB/s]\n",
"100% 137/137 [00:00<00:00, 850kiB/s]\n",
"100% 9.98G/9.98G [00:52<00:00, 190MiB/s]\n",
"100% 3.50G/3.50G [00:18<00:00, 194MiB/s]\n",
"100% 26.8k/26.8k [00:00<00:00, 73.6MiB/s]\n",
"100% 411/411 [00:00<00:00, 3.94MiB/s]\n",
"100% 1.84M/1.84M [00:00<00:00, 4.04MiB/s]\n",
"100% 500k/500k [00:00<00:00, 74.4MiB/s]\n",
"100% 649/649 [00:00<00:00, 6.04MiB/s]\n"
]
}
],
"source": [
"!python download-model.py TheBloke/StableBeluga2-GPTQ\n",
"!python download-model.py stabilityai/StableBeluga-7B"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "pIZVUMd_fLDn",
"outputId": "adb80642-a6f3-40d2-9958-572b7c4a9a0d"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Cloning into 'GPTQ-for-LLaMa'...\n",
"remote: Enumerating objects: 818, done.\u001b[K\n",
"remote: Counting objects: 100% (121/121), done.\u001b[K\n",
"remote: Compressing objects: 100% (21/21), done.\u001b[K\n",
"remote: Total 818 (delta 107), reused 100 (delta 100), pack-reused 697\u001b[K\n",
"Receiving objects: 100% (818/818), 469.73 KiB | 5.22 MiB/s, done.\n",
"Resolving deltas: 100% (497/497), done.\n",
"Collecting quant-cuda==0.0.0\n",
" Downloading https://github.com/jllllll/GPTQ-for-LLaMa-Wheels/raw/Linux-x64/quant_cuda-0.0.0-cp310-cp310-linux_x86_64.whl (465 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m465.0/465.0 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hInstalling collected packages: quant-cuda\n",
"Successfully installed quant-cuda-0.0.0\n"
]
}
],
"source": [
"# install GPTQ-for-LLaMa\n",
"!mkdir repositories\n",
"!cd repositories && git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda\n",
"!pip install https://github.com/jllllll/GPTQ-for-LLaMa-Wheels/raw/Linux-x64/quant_cuda-0.0.0-cp310-cp310-linux_x86_64.whl"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-jd20RTqawkq",
"outputId": "98e61d6e-8a21-4935-97bc-a36570d44832"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2023-07-28 13:55:10 WARNING:\u001b[33mThe gradio \"share link\" feature uses a proprietary executable to create a reverse tunnel. Use it with care.\u001b[0m\n",
"bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so\n",
"2023-07-28 13:55:13.122542: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
"2023-07-28 13:55:14 INFO:\u001b[32mLoading the extension \"gallery\"...\u001b[0m\n",
"Running on local URL: http://127.0.0.1:7860\n",
"Running on public URL: https://03a77c389e65bf8c55.gradio.live\n",
"\n",
"This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n",
"2023-07-28 13:55:49 INFO:\u001b[32mLoading stabilityai_StableBeluga-7B...\u001b[0m\n",
"2023-07-28 13:55:49 WARNING:\u001b[33mUsing the following 4-bit params: {'load_in_4bit': True, 'bnb_4bit_compute_dtype': torch.float16, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': False}\u001b[0m\n",
"2023-07-28 13:55:49 ERROR:\u001b[31mFailed to load the model.\u001b[0m\n",
"Traceback (most recent call last):\n",
" File \"/content/text-generation-webui/server.py\", line 68, in load_model_wrapper\n",
" shared.model, shared.tokenizer = load_model(shared.model_name, loader)\n",
" File \"/content/text-generation-webui/modules/models.py\", line 78, in load_model\n",
" output = load_func_map[loader](model_name)\n",
" File \"/content/text-generation-webui/modules/models.py\", line 189, in huggingface_loader\n",
" params['quantization_config'] = BitsAndBytesConfig(**quantization_config_params)\n",
" File \"/usr/local/lib/python3.10/dist-packages/transformers/utils/quantization_config.py\", line 118, in __init__\n",
" self.post_init()\n",
" File \"/usr/local/lib/python3.10/dist-packages/transformers/utils/quantization_config.py\", line 147, in post_init\n",
" raise ValueError(\n",
"ValueError: 4 bit quantization requires bitsandbytes>=0.39.0 - please upgrade your bitsandbytes version\n",
"\n",
"2023-07-28 13:56:12 INFO:\u001b[32mLoading stabilityai_StableBeluga-7B...\u001b[0m\n",
"Loading checkpoint shards: 100% 2/2 [01:18<00:00, 39.36s/it]\n",
"2023-07-28 13:57:40 WARNING:\u001b[33mmodels/stabilityai_StableBeluga-7B/tokenizer_config.json is different from the original LlamaTokenizer file. It is either customized or outdated.\u001b[0m\n",
"2023-07-28 13:57:40 INFO:\u001b[32mLoaded the model in 87.86 seconds.\n",
"\u001b[0m\n",
"2023-07-28 13:59:43 INFO:\u001b[32mProfile picture saved to \"cache/pfp_me.png\"\u001b[0m\n",
"2023-07-28 13:59:44 INFO:\u001b[32mProfile picture saved to \"cache/pfp_me.png\"\u001b[0m\n",
"Output generated in 8.23 seconds (1.58 tokens/s, 13 tokens, context 71, seed 594748621)\n",
"Output generated in 3.30 seconds (3.94 tokens/s, 13 tokens, context 71, seed 1954078381)\n",
"Output generated in 42.11 seconds (4.73 tokens/s, 199 tokens, context 276, seed 1286052067)\n",
"Output generated in 34.54 seconds (4.75 tokens/s, 164 tokens, context 517, seed 1419399526)\n",
"Output generated in 10.04 seconds (4.58 tokens/s, 46 tokens, context 697, seed 1689479419)\n",
"Output generated in 22.83 seconds (4.47 tokens/s, 102 tokens, context 697, seed 782675907)\n",
"2023-07-28 14:04:48 INFO:\u001b[32mSaved /content/text-generation-webui/characters/instruction-following/My Template.yaml.\u001b[0m\n",
"Output generated in 30.58 seconds (4.51 tokens/s, 138 tokens, context 1087, seed 1864039165)\n"
]
}
],
"source": [
"!python server.py --share --chat"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"machine_shape": "hm",
"provenance": [],
"authorship_tag": "ABX9TyM41QQidgHoa4edH9UiJ0jW",
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment