Download CircleCI Artifacts with directory structure

downloadArtifacts.sh

#!/usr/bin/env bash
#
# Download all artitacts for given build, mimicing structure locally.
#
# Batch size is slighly misleading, as it;s only how many URLs to attempt before momentary pause
#
# Can be used on CCI builds with parallelism and will only download the files for matcing index.
#
#
USAGE="$0 username/repo 123 [bitbucket] [batch_size]"



PROJECT=${1}  #user/project
BUILD=${2}  #1234
VCS=${3:-"github"}
BATCH_SIZE=${4:-100}

: ${CIRCLE_API_TOKEN:?"Please set CIRCLE_API_TOKEN as env variable"}
: ${BUILD:?"Please provide build number, usage:  $USAGE"}
case $PROJECT in
  *\/*)
    ;;
    *)
  echo -e "\nERROR: Invalid project format.\nusage: $USAGE"
  exit 1
esac

URL="https://circleci.com/api/v1.1/project/${VCS}/${PROJECT}/$BUILD/artifacts"
if [ -s steps.txt ];then
  echo "Using existing list, delete steps.txt to refresh"
else
  if [ "${CIRCLE_NODE_TOTAL:-1}" -gt "1" ];then
    echo "Running parallel nodes, only download files for index ${CIRCLE_NODE_INDEX}"
    curl -s -u $CIRCLE_API_TOKEN: $URL | jq -r --argjson node ${CIRCLE_NODE_INDEX} '.[] | select(.node_index==$node) | "mkdir -p $(dirname \(.node_index)/\(.path)) && curl -s \(.url | @sh) -o \(.node_index)/\(.path)" ' > steps.txt
  else
    curl -s -u $CIRCLE_API_TOKEN: $URL | jq -r '.[] | "mkdir -p $(dirname \(.node_index)/\(.path)) && curl -s \(.url | @sh) -o \(.node_index)/\(.path)" ' > steps.txt
  fi


fi

echo "Will download $(wc -l steps.txt) files"


current=0
backoff=2
while read cmd; do
   current=$(expr $current + 1)
   if [ $current -gt $BATCH_SIZE ];then
     echo "+$BATCH_SIZE.."
     current=0
     sleep $backoff
   fi
   eval $cmd &
done < steps.txt

Thanks for the script. I forked it because I found that running it required appending the $CIRCLE_API_TOKEN to each URL. I also wanted to download multiple builds so modified the steps.txt file name and local path to include the $BUILD variable.

Thanks for the script. I forked it because I found that running it required appending the $CIRCLE_API_TOKEN to each URL. I also wanted to download multiple builds so modified the steps.txt file name and local path to include the $BUILD variable.

Does it support CircleCI 2.x ?