Skip to content

Instantly share code, notes, and snippets.

@m-bartlett
Last active November 21, 2021 22:29
Show Gist options
  • Save m-bartlett/29220e1adaf8e6ddbf5e4b368e70689d to your computer and use it in GitHub Desktop.
Save m-bartlett/29220e1adaf8e6ddbf5e4b368e70689d to your computer and use it in GitHub Desktop.
Sparsely clone only a specific git repo directory using a GitHub tree URL

Uses git sparse checkout features to git clone only a given directory path within a repo.

Usage

Provide a GitHub tree-URL (a URL to a specific directory on a branch) as the only argument. Currently this only supports sparse checkout from GitHub.

Example: git clonedir https://github.com/GoogleChrome/chrome-extensions-samples/tree/mv3-migration-snippet/mv2-archive/api/messaging/timer

git Subcommand Note

If you have custom executables named as git-* in $PATH, git will automatically detect and offer these commands as subcommands. Therefore if you place git-clonedir in your $PATH (e.g. /usr/local/bin/) then you can also run it as git clonedir.

Clone strategy comparison and metrics

Metric Regular Clone git-clonedir
Time taken to clone ~12s
$ time git clone https://github.com/GoogleChrome/chrome-extensions-samples
Cloning into 'chrome-extensions-samples'...
remote: Enumerating objects: 12700, done.
remote: Counting objects: 100% (194/194), done.
remote: Compressing objects: 100% (126/126), done.
remote: Total 12700 (delta 84), reused 144 (delta 57), pack-reused 12506
Receiving objects: 100% (12700/12700), 52.25 MiB | 4.84 MiB/s, done.
Resolving deltas: 100% (6339/6339), done.

real    0m12.665s
user    0m3.657s
sys     0m1.274s
~2.5s
$ time git clonedir https://github.com/GoogleChrome/chrome-extensions-samples/tree/mv3-migration-snippet/mv2-archive/api/messaging/timer
Cloning into 'chrome-extensions-samples'...
remote: Enumerating objects: 6865, done.
remote: Counting objects: 100% (131/131), done.
remote: Compressing objects: 100% (89/89), done.
remote: Total 6865 (delta 58), reused 89 (delta 33), pack-reused 6734
Receiving objects: 100% (6865/6865), 964.59 KiB | 3.35 MiB/s, done.
Resolving deltas: 100% (3608/3608), done.
remote: Enumerating objects: 5, done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 5
Receiving objects: 100% (5/5), 1.71 KiB | 1.71 MiB/s, done.
Updating files: 100% (5/5), done.
Branch 'mv3-migration-snippet' set up to track remote branch 'mv3-migration-snippet' from 'origin'.
Switched to a new branch 'mv3-migration-snippet'

real    0m2.558s
user    0m0.302s
sys     0m0.116s
Repo file quantity 2563 files
$ find ./chrome-extensions-samples/ -type f | wc -l
2563
39 files
$ find ./chrome-extensions-samples/ -type f | wc -l
39
$ tree chrome-extensions-samples/
chrome-extensions-samples/
└── mv2-archive
    └── api
        └── messaging
            └── timer
                ├── clock.png
                ├── manifest.json
                ├── page.js
                ├── popup.html
                └── popup.js

4 directories, 5 files
Repo size 107MB
$ du -sh ./chrome-extensions-samples/
107M    ./chrome-extensions-samples/
1.7MB
$ du -sh ./chrome-extensions-samples/
1.7M    ./chrome-extensions-samples/
#!/usr/bin/env bash
set -o nounset
set -o errexit
git_url="$(
<<<"$1" sed -E 's%^(.*)((https://)|(git@))([^:/]+)[:/]([^/]+)/([^ ]+)(.*)(\.git)?%\2\5/\6/\7%g'
)"
IFS='/'
url_parts=($git_url)
protocol=${url_parts[0]}
host=${url_parts[2]}
user=${url_parts[3]}
repo=${url_parts[4]}
tree=${url_parts[5]}
branch=${url_parts[6]}
path=(${url_parts[@]:7})
url="$protocol//${host}/$user/$repo"
[ -d "$repo" ] || git clone --filter=blob:none --no-checkout "$url"
cd "$repo"
git sparse-checkout set "${path[*]}"
git checkout $branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment