veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

x1ppy

diff --git a/trunk/PKGBUILD b/trunk/PKGBUILD
index a7c3f0d..9abf71b 100644
--- a/trunk/PKGBUILD
+++ b/trunk/PKGBUILD
@@ -64,6 +64,8 @@ build() {
 		--with-pid-dir=/run \
 		--with-default-path='/usr/local/sbin:/usr/local/bin:/usr/bin' \
 
+    patch --strip=1 < ../../junest.patch
+

skye

# ==== Non-communicating processes

# 4x 1 chip per process:
os.environ["TPU_CHIPS_PER_PROCESS_BOUNDS"] = "1,1,1"
os.environ["TPU_PROCESS_BOUNDS"] = "1,1,1"
# Different per process:
os.environ["TPU_VISIBLE_DEVICES"] = "0" # "1", "2", "3"

# 1-liner for bash: TPU_CHIPS_PER_PROCESS_BOUNDS=1,1,1 TPU_PROCESS_BOUNDS=1,1,1 TPU_VISIBLE_DEVICES=0 TPU_MESH_CONTROLLER_ADDRESS=localhost:8476 TPU_MESH_CONTROLLER_PORT=8476

sshleifer

How BartConfig controls when LayerNorm is applied

6 groups of models inherit from BartForConditionalGeneration. The major differences between them are:

• pretraining objective & data
• finetuning objective & data
• number of layers and dimension of each layer
• when layernorm is applied

This document focuses on layernorm timing.

MarisaKirisame

import math

def sin(x):
    if isinstance(x, Dual):
        return Dual(sin(x.x), cos(x.x) * x.dx)
    return math.sin(x)

def cos(x):
    if isinstance(x, Dual):
        return Dual(cos(x.x), -1 * sin(x.x) * x.dx)

soxofaan

How to get a stack trace for each thread in a running Python script

Sometimes a Python script will simply hang forever with no indication of what is going wrong. Perhaps it's polling a service that will never return a value that allows the program to move forward.

Here's a way to see where the program is currently stuck, using pyrasite a tool for injecting code into running Python processes.

Install gdb and pyrasite

Install gdb.

jeasinema

#!/usr/bin/env python
# -*- coding:UTF-8 -*-

import torch
import torch.nn as nn
import torch.nn.init as init


def weight_init(m):
    '''

devhero

Byobu is a suite of enhancements to tmux, as a command line
tool providing live system status, dynamic window management,
and some convenient keybindings:

  F1                             * Used by X11 *
    Shift-F1                     Display this help
  F2                             Create a new window
    Shift-F2                     Create a horizontal split
    Ctrl-F2                      Create a vertical split
    Ctrl-Shift-F2                Create a new session

fatum12

#!/usr/local/bin/fontforge
# Usage: fontforge -script ttc2ttf.pe /path/to/font.ttc

fonts = FontsInFile($1)
n = SizeOf(fonts)
i = 0
while (i < n)
    Open($1 + "(" + fonts[i] + ")", 1)

    index = ToString(i + 1)

jnovack

Mosh (mobile shell) is a gift from the Gods(tm). Anyone with spotty internet or wireless connection has suffered the pain of a lost SSH session. Sure, one can fire up screen (or tmux as the kids are using these days), but that's an extra step and you are still using the SSH protocol.

I'm not here to tout the benefits of Mosh, you came here to open it up in your firewall.

1. Create the following file as /etc/firewalld/services/mosh.xml
2. firewall-cmd --add-service=mosh --permanent
3. firewall-cmd --reload

If you tend to have a lot of sessions (not recommended), you can increase the ports, but the default should be fine for most applications.