Dave Howcroft dmhowcroft

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

Base64 Patterns - Learning Aid

Base64 Code	Mnemonic Aid	Decoded*	Description
`JAB`	🗣 Jabber	`$.`	Variable declaration (UTF-16), e.g. `JABlAG4AdgA` for `$env:`
`TVq`	📺 Television	`MZ`	MZ header
`SUVY`	🚙 SUV	`IEX`	PowerShell Invoke Expression
`SQBFAF`	🐣 Squab favorite	`I.E.`	PowerShell Invoke Expression (UTF-16)
`SQBuAH`	🐣 Squab uahhh	`I.n.`	PowerShell Invoke string (UTF-16) e.g. `Invoke-Mimikatz`
`PAA`	💪 "Pah!"	`<.`	Often used by Emotet (UTF-16)

correlation metrics between character set and content language

data: Common Crawl September 2019 data set
SQL query on columnar index executed via AWS Athena

Git Wars: GitHub vs Bitbucket

Introduction

Now, you might think the answer I'm going to give you is already obvious because I'm using GiHub right now, but it's not. Both GitHub and Bitbucket offer great Git services, but each has its own features and pricing plans. In the following... thing, I'm going to compare the two and then offer a final solution that should work for most people.

TL;DR: Both. Use GitHub for open source and public repos (you'll spend most of your time here) and Bitbucket for private repos. But, sign up for GitHub first, then import account into Bitbucket. Also, check comments for updates. P.S. I personally prefer GitHub.

Interface and Functionality

The Best of the Best Practices (BOBP) Guide for Python

A "Best of the Best Practices" (BOBP) guide to developing in Python.

In General

Values

"Build tools for others that you want to be built for you." - Kenneth Reitz
"Simplicity is alway better than functionality." - Pieter Hintjens

One-line Tree in Python

Using Python's built-in defaultdict we can easily define a tree data structure:

def tree(): return defaultdict(tree)

That's it!

	%%-----------------------------------------------------------------------
	%% Make your own quadrille, graph, hex, etc paper!
	%% Uses the pgf/TikZ package for LaTeX, which should be part of
	%% any modern TeX installation.
	%% Email: [email protected]
	%% Twitter: @mcnees
	%%-----------------------------------------------------------------------

	\documentclass[11pt]{article}

	# Sane settings for Gnome
	gsettings set org.gnome.desktop.background show-desktop-icons true
	gsettings set org.gnome.desktop.interface clock-show-date true
	gsettings set org.gnome.settings-daemon.plugins.xsettings antialiasing 'grayscale'
	gsettings set org.gnome.settings-daemon.plugins.xsettings hinting 'slight'
	gsettings set org.gnome.desktop.interface text-scaling-factor '1.0'
	gsettings set org.gnome.desktop.interface monospace-font-name "Monospace 10"
	gsettings set org.gnome.desktop.interface document-font-name 'Sans 10'
	gsettings set org.gnome.desktop.interface font-name 'Cantarell 10'
	gsettings set org.gnome.desktop.interface gtk-theme 'Adwaita'

	Latency Comparison Numbers (~2012)
	----------------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD