cpursley

PostgreSQL is Enough

See the new site: https://postgresisenough.dev

---

• Simplify: move code into database functions
• Just Use Postgres for Everything
• Just use Postgres
• PostgreSQL is the worlds’ best database

veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

jiahao

# A port of https://github.com/karpathy/llama2.c/blob/master/run.c
# to Julia.
# Jiahao Chen <jiahao@csail.mit.edu> 2023-07-29
#
# MIT License: see full text at https://opensource.org/license/mit/
#


using LinearAlgebra
using LogExpFunctions

npilk

// Cloudflare Worker script to automatically redirect search queries based on trigger words

addEventListener("fetch", event => {
  event.respondWith(handleRequest(event.request))
})

// status code for redirect response; need something that won't cache
var statuscode = 303
    
// defining base URLs for search engines

todbot

# synthio_midi_synth.py - pretty usable MIDI-controlled synth using synthio in CircuitPython
# 11 May 2023 - @todbot / Tod Kurt
# Uses cheapie PCM5102 DAC on QTPY RP2040
# Video demo: https://www.youtube.com/watch?v=N-PbbWWDE6k
# Features:
# - midi velocity controls attack rate (gentle press = slow, hard press = fast)
# - notes have small random detune on all oscillators to reduce phase stacking
# - adjustable number of detuned oscillators per note 1-5 (midi controller 83)
# - five selectable waveforms: saw, squ, sin, noisy sin, noise (midi controller 82)
# - vibrato depth on mod wheel (midi controller 1)

yoavg

Reinforcement Learning for Language Models

Yoav Goldberg, April 2023.

Why RL?

With the release of the ChatGPT model and followup large language models (LLMs), there was a lot of discussion of the importance of "RLHF training", that is, "reinforcement learning from human feedback". I was puzzled for a while as to why RL (Reinforcement Learning) is better than learning from demonstrations (a.k.a supervised learning) for training language models. Shouldn't learning from demonstrations (or, in language model terminology "instruction fine tuning", learning to immitate human written answers) be sufficient? I came up with a theoretical argument that was somewhat convincing. But I came to realize there is an additional argumment which not only supports the case of RL training, but also requires it, in particular for models like ChatGPT. This additional argument is spelled out in (the first half of) a talk by John Schulman from OpenAI. This post pretty much

ingenieroariel

    Linus Torvalds (Creator of the Linux Kernel) - https://github.com/torvalds
    John Carmack (Co-founder of id Software) - https://github.com/ID_AA_Carmack
    Bjarne Stroustrup (Creator of C++) - https://github.com/BjarneStroustrup
    Fabrice Bellard (Creator of QEMU, FFMpeg, and Tiny C Compiler) - https://github.com/fbellard
    Andrei Alexandrescu (C++ expert and author) - https://github.com/incomputable
    Chandler Carruth (LLVM and Clang developer) - https://github.com/chandlerc
    Daniel Lemire (Computer science researcher, focuses on performance) - https://github.com/lemire
    P.J. Plauger - A renowned author, and contributor to the C Standard Library - https://github.com/pjplauger
    Peter J. Weinberger - Co-creator of AWK and a contributor to Unix - https://github.com/pjw
    Keith Packard - A prominent contributor to the X Window System, and the Linux graphics stack - https://github.com/keith-packard

jph00

Bing Chat at 2023-3-26 13:47:19

1

Q: Bing AI

2

Q: Some philosophers and AI researchers have claimed that AI can not be sentient, or be AGI, due to lack of "embodiment" or "grounding". Could you please summarize these claims, and points for and against? Who are the main people on each side of this debate?

rain-1

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

mattdesl

/**
 * A decoder for The "Quite OK Audio" (QOA) format, a lossy audio compression
 * that achieves relatively decent compression with fast decoding and not much complexity.
 *
 * Note that this has only been tested on QOA files generated by a specific commit (e8386f4)
 * of QOA, see below:
 *   https://github.com/phoboslab/qoa/tree/e8386f41d435a864ce2890e9f56d964215b40301
 *
 * If you try to decode audio files that were encoded on a different version of QOA you might
 * end up with jarring noise. This needs a companion encoder written in JS to be a little more stable.