Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active June 24, 2025 11:12
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@rain-1
rain-1 / LLM.md
Last active August 16, 2025 00:39
LLM Introduction: Learn Language Models

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

@kcalvinalvin
kcalvinalvin / utreexo-algorithms.md
Last active May 16, 2025 08:04
Utreexo algorithms

Add

Utreexo accumulator is a collection of proper merkle trees. All the merkle trees have 2 children or none.

Below is an example of adding 7 leaves to the tree. You can run the code I used to generate this at https://go.dev/play/p/um5263SDk59

This is an accumulator with 1 leaf. It has 1 root, which is also a leaf.

@NickCao
NickCao / riscv-gnu-toolchain.nix
Created December 8, 2022 06:59
riscv-gnu-toolchain.nix
{ lib
, stdenv
, fetchFromGitHub
, util-linux
, git
, cacert
, autoconf
, automake
, curl
, python3
import "dart:async";
import "dart:isolate";
import 'package:ferry/ferry.dart';
import 'package:ferry/ferry_isolate.dart';
import "package:gql_error_link/gql_error_link.dart";
import "package:gql_exec/gql_exec.dart";
import "package:gql_http_link/gql_http_link.dart";
import "package:gql_link/gql_link.dart";
import "package:gql_transform_link/gql_transform_link.dart";
# To set this up, first get tailscale working in an isolated linux shell:
# 1. sudo systemctl stop tailscaled.service
# 2. tailscaled -port 9993 -state tailscale-luks-setup.state -tun userspace-networking -socket ./tailscaled.sock
# 3. tailscale -socket ./tailscaled.sock up -hostname HOSTNAME-luks
# 4. tailscale -socket ./tailscaled.sock down
# 5. ctrl-c out of tailscaled
# 6 sudo systemctl start tailscaled.service
#
# Then add the .state file to your machine secrets and pass its path as tailscaleStatePath.
@kylebarron
kylebarron / convert.py
Last active August 31, 2024 04:55
preprocessing script for geoparquet on the web demo (https://observablehq.com/@kylebarron/geoparquet-on-the-web)
import geopandas as gpd
import pandas as pd
import pyarrow as pa
import pyarrow.feather as feather
import pyarrow.parquet as pq
import pygeos
import pyogrio
# https://ookla-open-data.s3.us-west-2.amazonaws.com/parquet/performance/type=mobile/year=2019/quarter=1/2019-01-01_performance_mobile_tiles.parquet
@eatonphil
eatonphil / psql-srv.py
Last active April 18, 2025 17:32 — forked from matteobertozzi/psql-srv.py
postgres "server" wire protocol example (ported python3)
# th30z@u1310:[Desktop]$ psql -h localhost -p 55432
# Password:
# psql (9.1.10, server 0.0.0)
# WARNING: psql version 9.1, server version 0.0.
# Some psql features might not work.
# Type "help" for help.
#
# th30z=> select foo;
# a | b
# ---+---

Normies just don't care about privacy

If you're a privacy enthusiast, you probably clicked a link to this post thinking it's going to vindicate you; that it's going to prove how you've been right all along, and "normies just don't care about privacy", despite your best efforts to make them care. That it's going to show how you're smarter, because you understand the threats to privacy and how to fight them.

Unfortunately, you're not right. You never were. Let's talk about why, and what you should do next.

So, first of all, let's dispense with the "normie" term. It's a pejorative term, a name to call someone when they don't have your exact set of skills and interests, a term to use when you want to imply that someone is clueless or otherwise below you. There's no good reason to use it, and it suggests that you're looking down on them. Just call them "people", like everybody else and like yourself - you don't need to turn them into a group of "others" to begin with.

Why does that matter? Well, would *y

@matinrco
matinrco / gitProxy.md
Last active June 15, 2025 07:10
Configure git/ssh to use socks/http proxy in windows, linux & wsl2

Windows

Add this to your ssh config file ( which is located in %USERPROFILE%\.ssh\config ) or create one if it's missing:

Http proxy & repo access via ssh:

Host *
    ProxyCommand "C:/Program Files/Git/mingw64/bin/connect.exe" -H {proxyserver}:{port} %h %p

or (for specific host name)

Host gitlab.com