Henning Thieß vroomfondel
vroomfondel
/ instructor_ollama_override.py
Last active
February 25, 2025 09:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import httpx | |
| import json | |
| from httpx import ByteStream | |
| from openai import OpenAI | |
| import instructor | |
| from pydantic import BaseModel | |
| from loguru import logger |
vroomfondel
/ startollama.sh
Created
May 25, 2024 14:57
start-script for running a locally built version of ollama but mounted into a docker container
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| cd $(dirname $0) || exit 1 | |
| OLLAMA_MODELDIR="${HOME}/ollama_models" | |
| OLLAMA_RUNDIR="${HOME}/ollama" | |
| OLLAMA_ALLOW_LOCAL_EXECUTABLE=1 | |
| OLLAMA_NUM_PARALLEL=2 | |
| echo $0 :: PWD: $(pwd) |
vroomfondel
/ build_me.sh
Created
May 25, 2024 14:56
ollama build script with patch for llama3 llama.cpp BPE pretokenization issue
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| cd $(dirname "$0") || exit 123 | |
| # builds a trimmed version of ollama (i.e. check the CPU flags and CUDA architectures below) | |
| # in my case: AMD Ryzen | RTX4090 | |
| # | |
| # to be put run in directory cloned from ollama and HEAD pointing to some tag | |
| # git clone [email protected]:ollama/ollama.git | |
| # in that cloned repo-dir: |