mvandermeulen

Complete Ollama Guide

Running GGUF Models Locally with Ollama

GGUF (GPT-Generated Unified Format) has quickly become the go-to standard for running large language models on your machine. There’s a growing number of GGUF models on Hugging Face, and thanks to community contributors like TheBloke, you now have easy access to them.

Ollama is an application based on llama.cpp that allows you to interact with large language models directly on your computer. With Ollama, you can use any GGUF quantized models available on Hugging Face directly, without the need to create a new Modelfile or download the models manually.

In this guide, we'll explore two methods to run GGUF models locally with Ollama:

mvandermeulen

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>KeepAlive</key>
    <true/>
    <key>Label</key>
    <string>hashicorp.consul.server</string>
    <key>ProgramArguments</key>
    <array>

mvandermeulen

How to use these files

in gnu-like systems

sed -f rm-en-wordlist-on-lines.sed corpus.list > output.file

should suffice, however on OSX, it may be required to run sed like this:

mvandermeulen

import base64
import hashlib
import hmac
import locale
import plistlib as plist
from datetime import datetime
import logging
import requests
import srp._pysrp as srp
import urllib3

mvandermeulen

import json
from dataclasses import MISSING, fields, is_dataclass
from inspect import isabstract, isclass
from types import GenericAlias, NoneType
from typing import Any, TypeAliasType, get_origin

import numpy as np
from numpy.typing import NDArray

type JSON = (

mvandermeulen

local M = {}

function M.make()
  local lines = {""}
  local winnr = vim.fn.win_getid()
  local bufnr = vim.api.nvim_win_get_buf(winnr)

  local makeprg = vim.api.nvim_buf_get_option(bufnr, "makeprg")
  if not makeprg then return end

mvandermeulen

# https://www.starlette.io/requests/
# https://developer.mozilla.org/ja/docs/Web/API/Streams_API/Using_readable_streams
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import HTMLResponse, StreamingResponse, Response
from logging import getLogger, StreamHandler
import asyncio
import tempfile
import json
import time
import os

mvandermeulen

from __future__ import annotations
from datetime import datetime
from typing import Any, Dict, List, Optional, AsyncGenerator
from sqlalchemy.dialects.postgresql import TIMESTAMP
from fastapi import Depends, FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field, ValidationError, validator
from pydantic_settings import BaseSettings
from sqlalchemy import JSON, Column, DateTime, Integer, String, func

mvandermeulen

from typing import List
from uuid import UUID
from sqlalchemy import create_engine
from sqlalchemy.orm import Session, sessionmaker
from fastapi import FastAPI, Depends, HTTPException
from sqlalchemy.orm import Session
from pydantic import BaseModel

class User(BaseModel):
    # This is a DB model - in SQLModel you can return the ORM model directly bc it's Pydantic under the hood

mvandermeulen

from fastapi import FastAPI, HTTPException, UploadFile, Form
from fastapi.middleware.cors import CORSMiddleware
from prisma import Prisma
from deepface import DeepFace
import numpy as np
from io import BytesIO
from PIL import Image, UnidentifiedImageError
import json
import os
import base64