For People In A Hurry: How to Set Up Open-WebUI with Ollama as free, local and privacy-focused alternative to ChatGPT

README.md

Note

This is a quick copy-paste-observe guide for people in a hurry to quickly set up Open-WebUI such that one can run and access a local Large Language Model (LLM) as a web application similar to OpenAI's ChatGPT.

Tip

This setup is recommended if your GPU hardware specifications falls below what is the typical minimum 64000 token context window required for agentic coding. Eg. Nvidia Geforce RTX 3060 with 12GB VRAM.

Instructions 📘

1. Install Ollama using the command below:

curl -fsSL https://ollama.com/install.sh | sh

2. Install Open-WebUI via docker using the command below:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

If you wish to utilise a Nvidia GPU, use the command below instead:

docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

3. Download ollama LLM model by using the command below:

ollama pull <model_name>
#Eg. ollama pull qwen3:8b

Warning

Note that, if you use any models with the cloud tag, it means it would not be fully local as it would be using Ollama's cloud models.

4. Access Open-WebUI via its dashboard:

Dashboard Link: http://localhost:3000

Note

Note that you can access it from an external network device via http://<SERVER_IP_ADDR>:3000.

References

1. Ollama Installation - https://ollama.com/
2. Open-WebUI Installation - https://github.com/open-webui/open-webui

Enable Web Search using SearXNG 🌏

Note

This is to avoid using proprietary Search Engine APIs where cost can easily go un-tracked and ballooned to staggering heights 💸💸💸 .
Using SearXNG, while requiring additional setup, allows for free and audit-able search actions.

Instructions 📘

cd $HOME

Create workspace for SearXNG:

mkdir searxng && cd searxng

Create settings.yml in the directory, searxng:

nano settings.yml

settings.yml

use_default_settings: true

general:
  debug: true

server:
  bind_address: "0.0.0.0"
  port: 8080
  limiter: false  # 🔥 disables rate limiting (important)

search:
  formats:
    - html
    - json

botdetection:
  enabled: false  # 🔥 disables bot blocking

Deploy SearXNG:

docker run -it -d \
	--name searxng_c \
	--restart unless-stopped \
	-p 8069:8080 \
	-v $(pwd)/settings.yml:/etc/searxng/settings.yml:ro \
searxng/searxng

Configure 🔧

Set the following properties in Open-WebUI, as seen in the screenshot below:

1. Ensure that Web Search Engine is set to searxng.
2. Ensure that Bypass Web Loader is enabled.

Note

The web loader is needed to bypass in order to avoid No sources given runtime error.

3. Set Searxng Query URL to the right url as seen in the screenshot.

Note

The Query URL may vary based on your setup but it should be as in the screenshot if you have set up the SearXNG docker above as instructed.

Verify ✔️

After setting up SearXNG docker, use the command below to verify it is working.

curl "http://localhost:8069/search?q=python&format=json"

After configuring in Open-WebUI, follow the steps below to verify that Open-WebUI can do web searches using SearXNG:

1. Create new Chat.

2. Toggle Web Search enabled for the Chat.

3. Enter the following prompt:

Search the web and tell me what is the latest exchange rate between SGD and Malaysia Ringgit. State the reference source link.

References 📚

1. Open-WebUI Web Search Troubleshooting