eggsyntax

Notes

• Point of confusion [resolved via email]: it's not clear to me that 'gradients of inputs wrt outputs' is actually new information on top of 'the model's outputs for all inputs'. Maybe I'm thinking too much in terms of LMs though?
• If we think about it in terms of something more continuous like image classification, the gradients are valuable in that they provide information about what parts of the input are most important, in the sense of "what inputs would, if tweaked, have the largest impact on the output" (for a specific case).
• In the limit, we can discover (and 'describe') everything about the model, eg by creating an enormous lookup table (by iterating over every possible input and recording the output produced, possibly with some additional complexity from tracking any additional internal state that the model has). This obviously isn't especially helpful for human-level understanding of a model's behavior, and would take an infeasible amount of time to create for any model large enough

eggsyntax

(ns common100k
  "Version 2, using 100k words instead of 10k"
  (:require [clojure.string :as s]))

(comment

  ;; words from https://github.com/first20hours/google-10000-english/blob/master/google-10000-english-usa.txt

  (def words (->> (slurp "./wiki-100k.txt")
                  (s/split-lines)

eggsyntax

'''
Implementation of the puzzle given at
https://worldspiritsockpuppet.com/2021/03/07/sleep-math-red-clay-blue-clay.html

I strongly suggest thinking about the puzzle for a while before looking at the
code; it's really interesting and counterintuitive, or was for me.

"Suppose that you have 1 kg of red clay that is 100 degrees and 1 kg of blue
clay that is 0 degrees. You can divide and recombine clay freely. If two pieces
of clay come into contact, temperature immediately equilibrates—if you put the

eggsyntax

shortname: ssc_archive
metadata:
  dc:
    title: SSC Archive
    creator: Scott Alexander
    language: en-US
tags:
  title: h2
content:
- "https://web.archive.org/web/20200217141740/https://slatestarcodex.com/2013/02/12/abraham-lincoln-ape-man/"

eggsyntax

(ns archiver
  (:require [clojure.string :as s]))

(def archive-root "https://web.archive.org/web/")

(def cutoff-date "20200621000000")

(def ssc-urls
  "Assumes the existence of a file in the current directory named 'ssc-urls' and
  containing a list of URLs, one per line."

eggsyntax

(ns ski.wotcha
  (:require [clojure.string :as string]))

;; Based on exercise at https://drive.google.com/drive/folders/1AQN08ikQZvq0QWn9KLuhiE9kJu1Tp2a5

;;; NB: The top-level code to actually solve the problem is down at the bottom:

;; Clojure relies on Java regex, which means regexes of any
;; complexity are pretty ugly:
(def name-regex #"^(\p{IsAlphabetic}+), (\p{IsAlphabetic}+)")

eggsyntax

(ns datapull
  "Datomic-ish pull for vanilla Clojure data structures. Usage examples in test below."
  (:require [clojure.test :as t]))

;; Alternatives:
;; - Juxt has a full-featured library for doing the same thing: https://github.com/juxt/pull
;; - Meander and Specter are both libraries for performing complex searches & transformations on Clojure data

(defn- seq-into
  "Like into, but returns nil if existing-seq is empty or contains only nils."

eggsyntax

# stuff I had to do on a fresh Ubuntu 18.04 install (minimal IIRC) to get the learning-from-human-preferences project to work:
# ( https://github.com/mrahtz/learning-from-human-preferences )

sudo apt install git
sudo apt install vim
mkdir bin
# Add /home/egg/bin and /home/egg/.local/bin to PATH
vim .bashrc
ln -s /usr/bin/python3 /home/egg/bin/python
python --version

eggsyntax

(ns shors.core
  (:require [clojure.math.combinatorics :as c]
            [clojure.test :as t :refer [is deftest]]))

(def tries (atom 0))

(defn test-val
  "Given p - 1, check whether p is a factor of n. Returns q if it is."
  [n p-1]
  (swap! tries inc)

eggsyntax

(ns tempdat
  "Minimal example of an unexpected Datomic behavior"
  (:require [datomic.api :as d]))

(def uri "datomic:mem://temp")
(d/delete-database uri) ; ensure fresh start
(d/create-database uri)

(def conn (d/connect uri))