Shawn Presser shawwn
shawwn
/ example.sh
Created
March 6, 2023 05:17
How I run 65B using my fork of llama at https://github.com/shawwn/llama
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mp=1; size=7B; # to run 7B | |
| mp=8; size=65B; # to run 65B | |
| for seed in $(randint 1000000) | |
| do | |
| export TARGET_FOLDER=~/ml/data/llama/LLaMA | |
| time python3 -m torch.distributed.run --nproc_per_node $mp example.py --ckpt_dir $TARGET_FOLDER/$size --tokenizer_path $TARGET_FOLDER/tokenizer.model --seed $seed --max_seq_len 2048 --max_gen_len 2048 --count 0 | tee -a ${size}_startrek.txt | |
| done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| I am Lieutenant Commander Data, and I am an android. | |
| I was created by Doctor Soong in the mid-2300s on Earth's moon colony. | |
| My positronic brain is a network of trillions of interconnected | |
| neurons that allow me to experience consciousness and sentience as | |
| only living beings can—and yet my mind operates at speeds far greater | |
| than those of most unenhanced organics. This makes it possible for me | |
| to perform complex analyses almost instantaneously while | |
| simultaneously running thousands of background processes without any | |
| decrease in efficiency or awareness. It also lets me communicate with |
shawwn
/ 65b_samples.txt
Last active
May 18, 2023 06:35
Some LLaMA 65B outputs after fixing the sampler settings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| {"seed": 374894, "temp": 0.7, "top_p": 0.0, "top_k": 40, "repetition_penalty": 1.1764705882352942, "max_seq_len": 512, "max_gen_len": 511} | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| Loading | |
| Loaded in 8.72 seconds | |
| ============== sample 1 ================= | |
| I believe the meaning of life is to grow, learn and give. |
shawwn
/ 65b.temp=0.7.top_p=0.0.top_k=40.repetition_penalty=
Created
March 6, 2023 02:58
Samples from LLaMA 65B. Settings:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| {"seed": 374894, "temp": 0.7, "top_p": 0.0, "top_k": 40, "repetition_penalty": 1.1764705882352942, "max_seq_len": 512, "max_gen_len": 511} | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| Loading | |
| Loaded in 8.72 seconds | |
| ============== sample 1 ================= | |
| I believe the meaning of life is to grow, learn and give. |
shawwn
/ llama_sizes.txt
Created
March 5, 2023 18:07
The size of each file distributed with LLaMA, for reference. See https://github.com/shawwn/llama-dl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ./tokenizer_checklist.chk 50 | |
| ./tokenizer.model 499723 | |
| ./7B/checklist.chk 100 | |
| ./7B/consolidated.00.pth 13476939516 | |
| ./7B/params.json 101 | |
| ./13B/checklist.chk 154 | |
| ./13B/consolidated.00.pth 13016334699 | |
| ./13B/consolidated.01.pth 13016334699 | |
| ./13B/params.json 101 | |
| ./30B/checklist.chk 262 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def lerp(a, b, t): | |
| return (b - a) * t + a | |
| @optimizer | |
| def adamsp(step_size=1e-1, b1=0.5): | |
| """Construct optimizer triple for AdamSP. | |
| Args: | |
| step_size: positive scalar, or a callable representing a step size schedule | |
| that maps the iteration index to a positive scalar (default 1e-1). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def lerp(a, b, t): | |
| return (b - a) * t + a | |
| def bias(i, x, beta): | |
| return 1 - jnp.asarray(beta, x.dtype) ** (i + 1) | |
| @optimizer | |
| def adam(step_size, b1=0.9, b2=0.999, eps=1e-8) -> OptimizerResult: | |
| """Construct optimizer triple for Adam. |
shawwn
/ hn_ignorance.js
Created
January 31, 2023 23:47
— forked from sillysaurus/hn_ignorance.js
HN Ignorance Is Bliss
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // ==UserScript== | |
| // @name HN Ignorance Is Bliss | |
| // @description Hide your comment scores and karma counters. See https://news.ycombinator.com/item?id=14456203 | |
| // @author sillysaurus3 | |
| // @version 1.0 | |
| // @match *://news.ycombinator.com/* | |
| // @grant none | |
| // @downloadURL https://gist.githubusercontent.com/sillysaurus/4d917e925548e4c7ec6f6bb96c94ef5c/raw | |
| // @updateURL https://gist.githubusercontent.com/sillysaurus/4d917e925548e4c7ec6f6bb96c94ef5c/raw | |
| // ==/UserScript== |
shawwn
/ hn_ignorance.js
Last active
February 1, 2023 00:05
— forked from sillysaurus/hn_ignorance.js
HN Ignorance Is Bliss
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // ==UserScript== | |
| // @name HN Ignorance Is Bliss | |
| // @description Hide your comment scores and karma counters. Installation instructions at https://news.ycombinator.com/item?id=14456203 | |
| // @author sillysaurus3 | |
| // @version 1.0 | |
| // @match *://news.ycombinator.com/* | |
| // @grant none | |
| // @downloadURL https://gist.githubusercontent.com/sillysaurus/4d917e925548e4c7ec6f6bb96c94ef5c/raw | |
| // @updateURL https://gist.githubusercontent.com/sillysaurus/4d917e925548e4c7ec6f6bb96c94ef5c/raw | |
| // ==/UserScript== |
shawwn
/ tpu_setup.sh
Last active
February 16, 2023 05:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # wget https://gist.githubusercontent.com/shawwn/88f64f7294c5a2e5e009d277a429ff2e/raw/tpu_setup.sh | |
| # bash tpu_setup.sh | |
| set -x | |
| pip3 install --upgrade pip | |
| # upgrade to nightly jax. | |
| pip3 install --force-reinstall --pre -U -f https://storage.googleapis.com/jax-releases/libtpu_releases.html 'jax[tpu]' 'jaxlib' | |
| pip3 install rich |