Andriy Oblivantsev eSlider

Gemma 4 edge GPU notes on local gfx90c (ROCm + Vulkan)

Date: 2026-08-01
Host: AMD Ryzen 7 5800H + Radeon Graphics (gfx90c / RADV RENOIR), ~12 GiB system RAM
Builds (llama.cpp 0005475):

TheRock HIP: build-work/llama.cpp/build-therock (GGML_HIP=ON, GPU_TARGETS=gfx90c, GGML_HIP_GRAPHS=OFF) — broken compute
Mesa Vulkan: build-work/llama.cpp/build-vulkan (GGML_VULKAN=ON, GGML_HIP=OFF) — working

Helpers: utils/bench-gpu.sh (HIP), utils/bench-vulkan.sh (Vulkan/RADV)

GPT-OSS on AMD Radeon 890M (llama.cpp)

Local inference for OpenAI gpt-oss-20b via llama.cpp and Vulkan.

Why gpt-oss-20b

Model	Fits this GPU?	Notes
gpt-oss-20b	✅ Yes	MXFP4 GGUF ~11.3 GiB; full GPU offload on 890M
gpt-oss-120b	❌ No	Needs ~80 GB VRAM (or heavy CPU offload + 64 GB+ RAM)

Кастомные модели в Cursor

Кратко: Cursor умеет подключать любой endpoint в формате OpenAI API (/v1/chat/completions). Локально это обычно Ollama, LM Studio или llama.cpp server — все они поднимают HTTP-сервер, а Cursor думает, что общается с OpenAI.

LLVM — это компиляторная инфраструктура (Clang, оптимизации), к LLM не относится. Скорее всего имелось в виду Ollama (или опечатка «Llama»).

1. Общая схема

GPT-OSS on AMD Radeon 890M (llama.cpp)

Local inference for OpenAI gpt-oss-20b via llama.cpp and Vulkan.

Why gpt-oss-20b

Model	Fits this GPU?	Notes
gpt-oss-20b	✅ Yes	MXFP4 GGUF ~11.3 GiB; full GPU offload on 890M
gpt-oss-120b	❌ No	Needs ~80 GB VRAM (or heavy CPU offload + 64 GB+ RAM)

OmniVoice Performance Benchmarks

Date: 2026-06-14
Project: /home/devops/projects/tts
Runtime: omnivoice.cpp (GGML / llama.cpp stack)
Model: k2-fsa/OmniVoice → GGUF (Serveurperso/OmniVoice-GGUF)

System

| Item | Value |


	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="UTF-8">
	<title>grok Chat Export</title>
	<style>
	body {
	font-family: Arial, sans-serif;
	max-width: 800px;

	-- Creates a function that is able to request any URL and return the response
	create or replace function get_http(url text, data json) returns text
	language plpython3u as
	$$
	import requests
	response = requests.get(url, verify=False, json=data)
	return response.text
	$$;

	-- Test-example call a webhook with data and get the response

	-- 1. Install
	-- apt-get update && apt-get install -f postgresql-plpython3-16 pip


	-- 2. Python extension
	CREATE EXTENSION plpython3u;

	-- 3. Install API python packages
	-- pip install http.client urllib3 --break-system-packages

	FROM postgres:16
	MAINTAINER Andriy Oblivantsev <eslider@gmail.com>

	# Set the environment variables
	ARG DEBIAN_FRONTEND=noninteractive
	ENV TZ "Europe/Madrid"
	ENV LANG en_US.UTF-8
	ENV LANGUAGE en_US:en
	ENV LC_ALL en_US.UTF-8