script to clone all repositories in a bitbucket server (aka stash) project

generate_clone_all_bitbucket_repos.sh

# this script uses syntax for bitbucket server. 
# For bitbucket cloud see https://confluence.atlassian.com/bitbucket/use-the-bitbucket-cloud-rest-apis-222724129.html
#
# Note: replace username, password and PROJECTNAME with your values
USERNAME=xxxx
PASSWORD=xxxx
PROJECTNAME=xxxxx

# jq syntax helpful links: 
# - http://stackoverflow.com/questions/30601278/how-to-extract-keys-from-multiple-nested-arrays-using-jq
# - http://stackoverflow.com/questions/18592173/select-objects-based-on-value-of-variable-in-object-using-jq

curl -s -u $USERNAME:$PASSWORD https://bitbucket.example.com/rest/api/1.0/projects/$PROJECTNAME/repos/ \
  | jq -r '.values[].links.clone[] | select(.name=="ssh") | .href' \
  | xargs -I {} echo "git clone '{}'" > clone_repos.sh

Thanks
this work fine if the number of repos are less than 25 , is there any way to get repos names if project contains more 25 repos .

if its more than 25 use https://bitbucket.example.com/rest/api/1.0/projects/$PROJECTNAME/repos/?limit\=1000

tx for the tip - I posted an updated version that works for both, bitbucket server and bitbucket cloud here: https://gist.github.com/eeichinger/c2eb46fbe7d5e2f49eba3bfaf5471759

I've built a cli for cloning and pulling git repos from bitbucket server
https://github.com/jensim/bitbucket_server_cli

I've built a cli for cloning and pulling git repos from bitbucket server
https://github.com/jensim/bitbucket_server_cli

Hey Jensim,
Is the cli you created for cloning all the repo's in bitbucket or does it clone repo's of a specific user?
I get the following error:
Clone/update all projects yes
Failed fetching user & project repos. Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 404 Not Found.'. Failed loading project repos due to 'Failed fetching projects from bitbucket, status code: 404 Not Found.'

@sreddy31 Are you trying it against bitbucket cloud repositories? The CLI only works for bitbucket server
That's the reason I didn't pick the name bitbucket_cli

If your problem is with a bitbucket server installation, please tell me what version of bitbucket server you are running, so that im able to look at the change logs, to determine if they have deprecated some of the API/1.0 endpoints for fetching users and projects.
Or have you configured non-standard paths ? You should be able to add such to the base-path when the CLI asks you for it..

@sreddy31 Are you trying it against bitbucket cloud repositories? The CLI only works for bitbucket server
That's the reason I didn't pick the name bitbucket_cli

If your problem is with a bitbucket server installation, please tell me what version of bitbucket server you are running, so that im able to look at the change logs, to determine if they have deprecated some of the API/1.0 endpoints for fetching users and projects.
Or have you configured non-standard paths ? You should be able to add such to the base-path when the CLI asks you for it..

@jensim, Many thanks for your reply.

I was adding a "/" at the end of my bitbucket URL, which tripped the program. The issue is now fixed.

I have found another rather interesting issue...
when I use the command:
bitbucket_server_cli clone --git-quiet --server https://bitbucket.myorg.mycompany.net --username bb_clone --password Password123 --all -H
the command fails with below:
etching projects [00:00:00] [########################################] 2/2 (eta:0s)
Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 401 Unauthorized.
Cause: Body: '{"errors":[{"context":null,"message":"You are not permitted to access this resource","exceptionName":"com.atlassian.bitbucket.AuthorisationException"}]}''
Working repos [00:00:00] [########################################] 2/2 (eta:0s)

however, if I export the same password as env variable, and run the command to grab the password from env variable, it runs without any issues.
export BITBUCKET_PASSWORD=Password123
bitbucket_server_cli clone --git-quiet --server https://bitbucket.myorg.mycompany.net --username test123 --env-password --all -H
Fetching users [00:00:11] [########################################] 948/948 (eta:0s)
Fetching projects [00:00:00] [########################################] 70/70 (eta:0s)
Working repos [00:00:04] [>---------------------------------------] 3/1606 (eta:22m)

Please could you help.

@sreddy31 Im glad it works

I think your problem might be with nbsp in the terminal you are using
I often trip on that the shell treats different whiteSpace differently
Hitting Space is not the same as Shift+Space
Have you tried sanitizing your input in some way
If you use any of JetBrains IDEs, they will highlight strange whiteSpace if you paste it into a scratch file..

@Jenism checked and double checked. I am using Ubuntu 18.04 LTS.
Same issue on my Mac too.
Fetching projects [00:00:00] [########################################] 2/2 (eta:0s)
Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 401 Unauthorized.'
Working repos [00:00:00] [########################################] 2/2 (eta:0s)

But, if I export the password as env variable and run the command, it works like a charm.

Your work has helped me save a lot of my time and I sincerely appreciate your efforts.

i have issues with this script.
then after some changes as below it works:
curl -D -X GET -H "Content-Type: application/json" https://$USERNAAME:$[email protected]/rest/api/1.0/projects/$PROJECTNAME/repos/
thanks.

@Jenism checked and double checked. I am using Ubuntu 18.04 LTS.
Same issue on my Mac too.
Fetching projects [00:00:00] [########################################] 2/2 (eta:0s)
Failed loading user repos due to 'Failed fetching users from bitbucket, status code: 401 Unauthorized.'
Working repos [00:00:00] [########################################] 2/2 (eta:0s)

But, if I export the password as env variable and run the command, it works like a charm.

Your work has helped me save a lot of my time and I sincerely appreciate your efforts.

Im really glad to hear it has helped another person out there 🥳
I created a issue on my project to track this issue
jensim/bitbucket_server_cli#21
Ill see to it as soon as i have some time to spare!

I've modified it a little bit.

curl -s -u "$ORG_USER:$ORG_PASS" "https://api.bitbucket.org/2.0/repositories/$ORG_TEAM?q=project.key%3D%22${ORG_PROJECT}%22&pagelen=100" \
| jq -r '.values[] | {slug:.slug, links:.links.clone[] } | select(.links.name=="ssh") | "git clone \(.links.href) \(.slug)"' | sort > clone-repos.sh

Make sure that you do not have more than 100 repos. In this case you need to add pagination, e.g. in a while-loop.

https://developer.atlassian.com/bitbucket/api/2/reference/meta/pagination

https://gist.github.com/gquere/fbf6fffd36565f15f98923ab6174a9c2

@sreddy31 mentioned wrong person :D typo probably, should have mentioned @jensim not me