carlosroman

New run - 2026-04-23 17:48:18

model	size	params	backend	ngl	n_batch	n_ubatch	type_k	type_v	fa	mmap	test	t/s
qwen35 27B bartowski:Q8_0	26.69 GiB	26.90 B	ROCm	999	1024	1024	q8_0	q8_0	1	0	pp512 @ d16384	288.15 ± 1.40
qwen35 27B bartowski:Q8_0	26.69 GiB	26.90 B	ROCm	999	1024	1024	q8_0	q8_0	1	0	tg128 @ d16384	7.10 ± 0.01
qwen35 27B bartowski:Q8_0	26.69 GiB	26.90 B	ROCm	999	1024	1024	q8_0	q8_0	1	0	pp512 @ d32768	228.82 ± 0.60
qwen35 27B bartowski:Q8_0	26.69 GiB	26.90 B	ROCm	999	1024	1024	q8_0	q8_0	1	0	tg128 @ d3

carlosroman

Setup llama.cpp

Clone git clone [email protected]:ggml-org/llama.cpp.git and then cd into llama.cpp and run the following:

uv venv --python 3.12
uv pip install --index-url https://repo.amd.com/rocm/whl/gfx1151/ "rocm[libraries,devel]"
source ./.venv/bin/activate
rocm-sdk init
deactivate

carlosroman

FROM python:3.11

WORKDIR /usr/src/app

COPY requirements.txt ./

RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

carlosroman

Simple test to show TLS over UDS

To run:

go test -v -timeout 1s

The third test hangs (ClientUsesTLS) at the moment. The output does look like the client has encrypted the data as it got sent to the listner.

carlosroman

# syntax=docker/dockerfile:1

# Allows to cheange the JDK image used
ARG JRE_DOCKER_IMAGE=eclipse-temurin:11

# Use the official JDK image as the base image
FROM ${JRE_DOCKER_IMAGE}

WORKDIR /app

carlosroman

---
version: "3.9"
services:
  zookeeper:
    image: quay.io/debezium/zookeeper:2.3
    ports:
     - 2181:2181
     - 2888:2888
     - 3888:3888
  kafka:

carlosroman

Simple HA proxy with fallback backend

To run:

docker compose up # or docker-compose up if not using the docker plugin

In another terminal the run the following:

carlosroman

# syntax=docker/dockerfile:1

FROM python:3.11-alpine

COPY random-logger.py /usr/bin/

ENTRYPOINT [ "python" ]

CMD [ "/usr/bin/random-logger.py" ]

carlosroman

---

- name: Create/update CloudFront origin access identity
  command: aws cloudfront create-cloud-front-origin-access-identity --cloud-front-origin-access-identity-config "CallerReference={{ site }},Comment=access-identity-{{ site }}.site.s3.amazonaws.com"
  environment:
    AWS_ACCESS_KEY_ID: "{{ aws_access_key }}"
    AWS_SECRET_ACCESS_KEY: "{{ aws_secret_access_key }}"
    AWS_DEFAULT_REGION: "{{ aws_region }}"
  no_log: true
  register: aws_cli_out

carlosroman

type SomeResource struct {
	counter int
}

func (sr *SomeResource) incCounter(rw http.ResponseWriter, r *http.Request) {
	sr.counter++
}

func (sr *SomeResource) getCounter(rw http.ResponseWriter, r *http.Request) {
	v := strconv.Itoa(sr.counter)