GitHub Organization Repository Cloner

This script provides a convenient way to clone all repositories from a GitHub organization with just one command. It includes detailed progress information and error handling.

Features

Clones all repositories from a specified GitHub organization
Shows detailed progress information
Handles errors gracefully
Skips existing repositories
Counts and displays the total number of repositories
Provides two implementations: one using jq (recommended) and one without dependencies

Requirements

For the recommended version: jq needs to be installed
For the alternative version: only standard Unix tools are required

Installation

Copy the script below into your ~/.zshrc file (or ~/.bashrc if you use Bash)
Reload your shell configuration: source ~/.zshrc

# Function to clone all repositories from a GitHub organization
CloneAll() {
    if [ -z "$1" ]; then
        echo "Error: No organization name provided."
        echo "Usage: CloneAll <github-organization-name>"
        return 1
    fi
    
    echo "Starting to clone all repositories from organization: $1"
    
    # Make the url to the input github organization's repository page.
    ORG_URL="https://api.github.com/orgs/${1}/repos?per_page=200"
    echo "Fetching repository list from: ${ORG_URL}"
    
    # Use jq to properly extract the repository URLs from JSON
    if ! command -v jq &> /dev/null; then
        echo "Error: This function requires jq to be installed."
        echo "Please install jq and try again."
        return 1
    fi
    
    # Get repository URLs using jq to ensure proper JSON parsing
    echo "Extracting repository URLs..."
    ALL_REPOS=$(curl -s "${ORG_URL}" | jq -r '.[] | .html_url')
    
    # Check if we got any repositories
    if [ -z "$ALL_REPOS" ]; then
        echo "Error: No repositories found or organization doesn't exist."
        return 1
    fi
    
    # Count repositories
    REPO_COUNT=$(echo "$ALL_REPOS" | wc -l | tr -d ' ')
    echo "Found ${REPO_COUNT} repositories to clone."
    
    # Clone all the repositories.
    COUNTER=1
    echo "$ALL_REPOS" | while read -r ORG_REPO; do
        echo "[$COUNTER/$REPO_COUNT] Cloning repository: ${ORG_REPO}"
        REPO_NAME=$(basename "${ORG_REPO}")
        
        if [ -d "$REPO_NAME" ]; then
            echo "Directory $REPO_NAME already exists, skipping."
        else
            echo "Running: git clone ${ORG_REPO}"
            git clone "${ORG_REPO}"
            if [ $? -eq 0 ]; then
                echo "Successfully cloned ${ORG_REPO}"
            else
                echo "Failed to clone ${ORG_REPO}"
            fi
        fi
        
        COUNTER=$((COUNTER+1))
        echo "----------------------------------------"
    done
    
    echo "All done! Cloned repositories from $1."
}

# Alternative implementation without jq dependency
CloneAllNoJQ() {
    if [ -z "$1" ]; then
        echo "Error: No organization name provided."
        echo "Usage: CloneAllNoJQ <github-organization-name>"
        return 1
    fi
    
    echo "Starting to clone all repositories from organization: $1"
    
    # Make the url to the input github organization's repository page.
    ORG_URL="https://api.github.com/orgs/${1}/repos?per_page=200"
    echo "Fetching repository list from: ${ORG_URL}"
    
    # Extract repository URLs more carefully
    echo "Extracting repository URLs..."
    RESPONSE=$(curl -s "${ORG_URL}")
    ALL_REPOS=$(echo "$RESPONSE" | grep -o '"html_url": *"[^"]*"' | cut -d'"' -f4 | grep -v "api.github.com")
    
    # Check if we got any repositories
    if [ -z "$ALL_REPOS" ]; then
        echo "Error: No repositories found or organization doesn't exist."
        return 1
    fi
    
    # Count repositories
    REPO_COUNT=$(echo "$ALL_REPOS" | wc -l | tr -d ' ')
    echo "Found ${REPO_COUNT} repositories to clone."
    
    # Clone all the repositories.
    COUNTER=1
    echo "$ALL_REPOS" | while read -r ORG_REPO; do
        echo "[$COUNTER/$REPO_COUNT] Cloning repository: ${ORG_REPO}"
        REPO_NAME=$(basename "${ORG_REPO}")
        
        if [ -d "$REPO_NAME" ]; then
            echo "Directory $REPO_NAME already exists, skipping."
        else
            echo "Running: git clone ${ORG_REPO}"
            git clone "${ORG_REPO}"
            if [ $? -eq 0 ]; then
                echo "Successfully cloned ${ORG_REPO}"
            else
                echo "Failed to clone ${ORG_REPO}"
            fi
        fi
        
        COUNTER=$((COUNTER+1))
        echo "----------------------------------------"
    done
    
    echo "All done! Cloned repositories from $1."
}

Usage

Using the jq version (recommended)

# Clone all repositories from the 'microsoft' organization
CloneAll microsoft

Using the version without jq

# Clone all repositories from the 'google' organization
CloneAllNoJQ google

Example Output

Starting to clone all repositories from organization: keybase
Fetching repository list from: https://api.github.com/orgs/keybase/repos?per_page=200
Extracting repository URLs...
Found 25 repositories to clone.
[1/25] Cloning repository: https://github.com/keybase/client
Running: git clone https://github.com/keybase/client
Cloning into 'client'...
remote: Enumerating objects: 380244, done.
remote: Total 380244 (delta 0), reused 0 (delta 0), pack-reused 380244
Receiving objects: 100% (380244/380244), 273.92 MiB | 10.25 MiB/s, done.
Resolving deltas: 100% (287173/287173), done.
Successfully cloned https://github.com/keybase/client
----------------------------------------
[2/25] Cloning repository: https://github.com/keybase/go-keybase-chat-bot
...

Installing jq

If you want to use the recommended version with jq, you can install it using your package manager:

On macOS with Homebrew:

brew install jq

On Ubuntu/Debian:

sudo apt-get install jq

On Fedora:

sudo dnf install jq

On Arch Linux:

sudo pacman -S jq

Troubleshooting

Authentication Issues: If you encounter authentication issues, you might need to set up a GitHub token.
Rate Limiting: GitHub API has rate limits. If you hit them, the script will show an error.
Large Organizations: For organizations with more than 200 repositories, modify the per_page=200 parameter.

ant4g0nist/github-org-cloner.md