wsl2 journey

wsl2-scratchpad.md

Journey to the center of WSL2 Ubuntu

ssh matters

Host OS: ensure file permissions comply with ssh requirements.

$ find /mnt/c/Users/vmarch/Documents/.ssh -printf "%M %p\n"  
drwx------ /mnt/c/Users/$USER/Documents/.ssh
-rw------- /mnt/c/Users/$USER/Documents/.ssh/authorized_keys
-rw------- /mnt/c/Users/$USER/Documents/.ssh/config
-r-------- /mnt/c/Users/$USER/Documents/.ssh/id_ed25519
-r-------- /mnt/c/Users/$USER/Documents/.ssh/id_ed25519.pub
-rw------- /mnt/c/Users/$USER/Documents/.ssh/known_hosts

WSL2 OS symlinks select individual files. Don't symlink the whole .ssh/ directory, it won't work!

$ tree ~/.ssh/
~/.ssh/
├── authorized_keys
├── config -> /mnt/c/Users/$USER/Documents/.ssh/config
├── id_ed25519
├── id_ed25519.pub -> /mnt/c/Users/$USER/Documents/.ssh/id_ed25519.pub
└── known_hosts -> /mnt/c/Users/$USER/Documents/.ssh/known_hosts

HF cache -- /mnt/c/... is a big no no

Avoid putting HF cache on WSL2 /mnt/c/..., as WSL2 has slow cross-OS access (WSL2 doc, SO). It slows down loading checkpoint -- very noticable even with a 2.7b model (phi-2). Excluding the dir on Windows Defender does not help.

# HF Cache on /mnt/c/Users/$USER/AppData/Roaming/JetBrains/cache/huggingface/
$ python haha/haha.py
Loading checkpoint shards: 100%|████████████████████████████████████████| 2/2 [04:59<00:00, 149.60s/it]

# HF Cache on ~/.cache/huggingface/
$ python haha/haha.py
Loading checkpoint shards: 100%|████████████████████████████████████████| 2/2 [00:04<00:00,  2.36s/it]