Preetam Patil ityogi

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

How to send bulk/mass email

The short answer is that you need to set up your own web server (Hetzner, AWS, DigitalOcean, etc.), install email software on it (Listmonk, Mailwizz, Mautic), and use an SMTP like Amazon SES. It's not that hard. If you're on Windows, Putty and FileZilla will be your main programs to access your server. When using CSV files for your contacts, you want to use UTF-8 format.

There are some detailed guides below for Sendy and Listmonk. But even if you have/want to hire someone to set it up for you, they should be able to do so for under $60 (check Fiverr). So it's still the most affordable option.

When I searched for this I had a very hard time finding a right answer because all the results were SEO blogs advertising their newsletter services (Mailchimp, Convertkit, etc.), which is not the same thing.

My use case is that I have a

FAQ on the xz-utils backdoor (CVE-2024-3094)

This is a living document. Everything in this document is made in good faith of being accurate, but like I just said; we don't yet know everything about what's going on.

Update: I've disabled comments as of 2025-01-26 to avoid everyone having notifications for something a year on if someone wants to suggest a correction. Folks are free to email to suggest corrections still, of course.

Background

A Short IPv6 Guide for Home IPv4 Admins

This guide is for homelab admins who understand IPv4s well but find setting up IPv6 hard or annoying because things work differently. In some ways, managing an IPv6 network can be simpler than IPv4, one just needs to learn some new concepts and discard some old ones.

Let’s begin.

First of all, there are some concepts that one must unlearn from ipv4:

Concept 1

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

	#!/bin/bash

	# whisper-stream.sh
	#
	# Take a url supported by yt-dlp, dump 30-second segments to the current
	# directory named by unix timestamp, and transcribe each segment using Whisper.
	#
	# example: TZ=Australia/Canberra ./whisper-stream.sh "https://..."
	#
	# The time displayed is the time when ffmpeg first opens the segment for

	# Power Usage Calculator for AI Workloads

	'''
	# Serving
	$ vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 4 --num-scheduler-steps 20 --quantization=fp8 --gpu-memory-utilization=0.97
	INFO 01-13 04:59:05 api_server.py:712] vLLM API server version 0.6.6.post2.dev5+g5ce4627a

	# Benchmark - we do bs=64 to emulate https://arxiv.org/pdf/2310.03003
	cmd = [
	"python", os.path.expanduser("~/vllm/benchmarks/benchmark_serving.py"),