Local LLM cheat-sheet

local_llm.md

OLLAMA

Install

curl -fsSL https://ollama.com/install.sh | sh

Download model

ollama pull <model>

For example:

ollama pull qwen3:8b
ollama pull mistral

List available models:

 ollama list

Run local model

ollama run mistral "Say hello"

Check server

By default, the server is available at:

http://localhost:11434

Query API

From Python:

import requests

response = requests.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "qwen3:8b",
        "prompt": "Explain quantum computing in one paragraph."
    }
)

Using OpenAI-compatible endpoint:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="dummy"
)

response = client.chat.completions.create(
    model="qwen3:8b",
    messages=[
        {"role": "user", "content": "Hello"}
    ]
)

print(response.choices[0].message.content)

query_local_ai.py

#!/usr/bin/env python3
import json
import requests
import time
import argparse

def query_model(model, prompt):
	response = requests.post(
	    "http://localhost:11434/api/chat",
	    json={
		"model": model,
		"messages": [
		    {
		        "role": "user",
		        "content": prompt
		    }
		],
		"options": {
		    "num_ctx": 4096
		},
		"stream": False
	    }
	)
	data = response.json()
	return data["message"]["content"]

def main():
	parser = argparse.ArgumentParser(description="JSCeal deobfuscator: request the function name via LLM.")
	parser.add_argument("--model", default="mistral", help="Ollama model")
	parser.add_argument("--prompt", default="What is 2+2? Answer with one word.", help="Prompt for the LLM")
	args = parser.parse_args()
	
	start = time.perf_counter()
	resp = query_model(args.model, args.prompt)
	elapsed = time.perf_counter() - start
	print(resp)
	print(f"\nElapsed: {elapsed:.3f} seconds")

if __name__ == "__main__":
	main()