Skip to content

Instantly share code, notes, and snippets.

@Genzer
Last active March 16, 2024 02:35
Show Gist options
  • Save Genzer/c96f98da8c562da0e0e0ad9b792cb7dc to your computer and use it in GitHub Desktop.
Save Genzer/c96f98da8c562da0e0e0ad9b792cb7dc to your computer and use it in GitHub Desktop.
Calculate total storage of Git repositories on GitHub
#!/usr/bin/env bash
set -u -e -o pipefail
# INTRODUCTION
# ---
#
# This snippet it aggregrate the `size` of all repositories in an GitHub Orginazation
# to report the total used storage (approximately).
#
# PREREQUISITES
# ---
#
# - Bash 5.0+
# - GitHub CLI
# - jq
# - numfmt (optional)
# - GITHUB_TOKEN is available or GitHub CLI is authenticated
# - Access to all repositories of the GitHub Organization.
#
# If you are using macOS, you need to install `coreutils`
# (e.g use `brew install coreutils`) and make `numfmt` available in `$PATH`.
# [Thanks to @slgraff!)
readonly org="$1"
gh api -X GET --paginate "orgs/$org/repos" `# (1)` \
| jq '.[] | .size' | jq -s '. | add' `# (2)` \
| numfmt --from-unit=1K --to=si `# (3)`
# IMPLEMENTATION NOTES
# ----
#
# (1) The GitHub CLI `api` is used becauses `gh repos list` does not provide the size attribute.
# Also `--paginate` is used to make sure that all repositories are fetched.
# It's important that GitHub CLI, as of 2.20.2, will produce a multiple JSON arrays instead of just one.
# This is a known issue https://github.com/cli/cli/issues/1268.
#
# (2) Using `jq` to extract only the `size` (Note that GitHub API reports the size in Kilobyte [1]- SI Unit).
# Because of the issue of `--paginate`, a second `jq` step is required to use "--slurp/-s" to join all the results
# into one array. Finally, all elements are aggregated using `add` to sum them up.
#
# (3) This step is using `numfmt` to convert the size into something human-readable.
#
# ----
# [1]: https://stackoverflow.com/questions/14090214/what-does-the-parameter-size-mean-in-github-repo-information
@slgraff
Copy link

slgraff commented Mar 13, 2024

On macOS you will need to use gnumfmt which is included in coreutils.
brew install coreutils

@Genzer
Copy link
Author

Genzer commented Mar 16, 2024

Hi @slgraff, thank you!

I was using macOS at the time and I did installed coreutils to have numfmt. I updated the gist to incorporate this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment