Skip to content

Instantly share code, notes, and snippets.

View SinclairCoder's full-sized avatar
🎯
Focusing

Zengzhi Wang SinclairCoder

🎯
Focusing
148 followers · 865 following
View GitHub Profile
@SinclairCoder
SinclairCoder / README_hfd.md
Created November 11, 2023 12:02 — forked from padeoe/README_hfd.md
Command-line Tool for Easy Downloading of Huggingface Models

🤗Huggingface Model Downloader

Update: The previous version has a bug. When resuming from a breakpoint, there may be an issue causing incomplete files. Please update to the latest version!!!

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer， this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

  • ⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
  • 🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
  • 🚫 File Exclusion: Use --exclude to skip specific files, save time for models with duplicate formats (e.g., .bin and .safetensors).
  • 🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.