Clone all repositories of a Github user

clone.bash

curl -s https://api.github.com/users/milanboers/repos | grep \"clone_url\" | awk '{print $2}' | sed -e 's/"//g' -e 's/,//g' | xargs -n1 git clone

I encountered the same issue and made this helper has a bunch of other useful utilities too for managing your github projects

((page_count = public_repos / 100 + 1))

+10 for this smart move, you fetched the API repo amount and then use it for pagination to expand the crawl limits.

Will try your tool at some time, atm no ongoing tasks for that. But still good job! 👍

In my case, the pages have 30 entries, not 100. In any case, I think this solution is independent of that:

gubAll() {
  page=1
  repos=0
  entries=`curl -s https://api.github.com/users/$1/repos?page=$page | grep \"clone_url\"`
  while [[ -n $entries ]]
  do
    # echo "$entries"
    repos=$(( $repos + $(echo "$entries" | wc -l) ))
    echo "$entries" | awk '{print $2}' | sed -e 's/"//g' -e 's/,//g' | xargs -n1 git clone
    page=$(( $page + 1 ))
    entries=`curl -s https://api.github.com/users/$1/repos?page=$page | grep \"clone_url\"`
  done
  echo Pages: $(( $page -1 )), repos: $repos.
}

Then:

$ gubAll milanboers

will grab all 32 repos (at this point in time) from milanboers.

This simultaneously downloads as many repos as possible from any organization using parallel

ORGANIZATION=myorg
NUM_REPOS=2000

gh repo list $ORGANIZATION -L $NUM_REPOS --json name |
    jq -r --arg org $ORGANIZATION '$org + "/" + .[].name' |
    parallel gh repo clone