Clone all repos from a GitHub organization

clone-all-twitter-github-repos.sh

curl -s https://api.github.com/orgs/twitter/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["ssh_url"]} ]}'

• | xargs -n1 git clone

we can essentially remove jq and directly use gh

gh repo list --json=sshUrl --limit 1000 -q ".[].sshUrl" | xargs -n1 git clone

though, it's asking for ssh passcode... any idea how to overcome that?

@Vaisakhkm2625,

If you don't want to use SSH protocol, just run this command¹:

gh auth login

It's interactive, so there are a couple of selections you need to make first i.e., selecting the host (GitHub or GitHub Enterprise) etc. but eventually you will reach:

What is your preferred protocol?
> http
  ssh

Select ssh, and it gh will save your preference for future gh commands. I'd recommend doing it this way so that gh can manage your credentials, BUT you can probably skip all the above by running²:

gh config set git_protocol ssh

Oops, typo! Instead of the one-liner immediately above, to use `http` instead of `ssh` the command should read

"gh config set git_protocol ssh http" (see diff below).

- gh config set git_protocol ssh
# use the below for http protocol (to bypass ssh requirement)
+ gh config set git_protocol http

Doy! 😄

Footnotes

1. gh auth login (cli.github.com/manual) ↩

2. gh config set (cli.github.com/manual) ↩

I just want to say that this thread is a great example of the open source community. An 8-year thread on a platform, with everybody posting little bits of code and share with others for free, on the platform itself.

I agree.

To clone all repos from an organisation using HTTP

gh auth login
gh repo list <organisation> --json=url --limit 1000 -q ".[].url" | xargs -n1 git clone

1- Install jq, a lightweight and flexible command-line JSON processor
2- Fetch a list of repositories from the specified GitHub organization and clone them

brew install jq

curl -s https://api.github.com/orgs/<ORG_NAME>/repos?per_page=100 | jq '.[].html_url' | xargs -n 1 git clone --depth=1curl -s https://api.github.com/orgs/<ORG_NAME>/repos\?per_page\=100 | jq '.[].html_url' | xargs -n 1 git clone --depth=1

Replace <ORG_NAME> with the actual name of the GitHub organization

• Use curl to retrieve the repository data from the GitHub API with a limit of 100 repositories per page
• Pipe the result to jq to extract the HTML URLs of all repositories
• Use xargs to clone each repository URL to the local machine with a shallow copy (depth=1), which retrieves only the latest commit history

• | xargs -n1 git clone

we can essentially remove jq and directly use gh

gh repo list --json=sshUrl --limit 1000 -q ".[].sshUrl" | xargs -n1 git clone

though, it's asking for ssh passcode... any idea how to overcome that?

@Vaisakhkm2625,

If you don't want to use SSH protocol, just run this command1:

gh auth login

It's interactive, so there are a couple of selections you need to make first i.e., selecting the host (GitHub or GitHub Enterprise) etc. but eventually you will reach:

What is your preferred protocol?
> http
  ssh

Select ssh, and it gh will save your preference for future gh commands. I'd recommend doing it this way so that gh can manage your credentials, BUT you can probably skip all the above by running2:

gh config set git_protocol ssh

Oops, typo! Instead of the one-liner immediately above, to use http instead of ssh the command should read

"gh config set git_protocol ssh **http**" (see diff below).

- gh config set git_protocol ssh
# use the below for http protocol (to bypass ssh requirement)
+ gh config set git_protocol http

Doy! 😄

Footnotes

1. [`gh auth login` (cli.github.com/manual)](https://cli.github.com/manual/gh_auth_login) [↩](#user-content-fnref-1-9e7baee31a028e56fce81af59af26625)

2. [`gh config set` (cli.github.com/manual)](https://cli.github.com/manual/gh_config_set#:~:text=%24%20gh%20config%20set%20git_protocol%20ssh%20%2D%2Dhost%20github.com) [↩](#user-content-fnref-2-9e7baee31a028e56fce81af59af26625)

got around by using ssh agent

eval "$(ssh-agent -s)"  # Start the ssh-agent
ssh-add ~/.ssh/id_rsa    # Add your SSH key:
gh repo list --json=sshUrl --limit 1000 -q ".[].sshUrl" | xargs -n1 git clone