christophemarois

# on the remote

mkdir my-repo
cd my-repo
git init
git config receive.denyCurrentBranch updateInstead

# on the local

git remote add [remote-name] ssh://[ssh profile name]/[path to repo on the remote]

jimsrc

Overview

I just read this trick for text compression, in order to save tokens in subbsequent interactions during a long conversation, or in a subsequent long text to summarize.

SHORT VERSION:

It's useful to give a mapping between common words (or phrases) in a given long text that one intends to pass later. Then pass that long text to gpt-4 but encoded with such mapping. The idea is that the encoded version contains less tokens than the original text. There are several algorithms to identify frequent words or phrases inside a given text, such as NER, TF-IDF, part-of-speech (POS) tagging, etc.