Skip to content

Instantly share code, notes, and snippets.

@xvdp
Last active June 14, 2023 23:39
Show Gist options
  • Save xvdp/2d093ed95ca109a7823c9304230ea236 to your computer and use it in GitHub Desktop.
Save xvdp/2d093ed95ca109a7823c9304230ea236 to your computer and use it in GitHub Desktop.
Set DL model caches to a shared mount or away from $HOME/.caches
#!/bin/bash
# change cache paths for huggingface, torch, openai (uses ~/.cache/ by default)
# use: set_model_caches.sh /mnt/my/model/cache/path
cache_path=$1
# Check if the argument is provided
if [ -z "$cache_path" ]; then
echo "Error: Cache path argument is missing."
echo "Usage: $0 /path_to_cache"
exit 1
fi
#
# back up .bashrc
#
bashrc_path=~/.bashrc
bashrc_backup=~/.bashrc_backup_$(date +"%Y%m%d_%H%M%S")
cp "$bashrc_path" "$bashrc_backup"
i=0
names=(HUGGINGFACE_HOME TORCH_HOME XDG_CACHE_HOME)
paths=("${cache_path}/huggingface" "${cache_path}/torch" $cache_path)
# defaults=(~/.cache/huggingface ~/.cache/torch ~/.cache)
for name in "${names[@]}"; do
path=${paths[i]}
existing_line="export ${name}="
if grep -qF "$existing_line" "$bashrc_path"; then # Check if HUGGINGFACE_HOME already exists in .bashrc
sed -i "s|$existing_line.*|$existing_line$path|" "$bashrc_path"
else
echo "$existing_line$path" >> "$bashrc_path"
fi
echo $name $path
i=$((i + 1))
done
# Reload the .bashrc file
source "$bashrc_path"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment