Download entire iCloud shared albums

icloud-album-download.sh

#!/bin/bash

# Description: Downloads Web Albums shared by Apple Photos
# Requirements: jq
# Usage: ./icloud-album-download.sh <URL> [<target folder>]
# Source: https://gist.github.com/AlexanderWillner/b8124af1979e88d4046987c953b8260f
# Author: @zneak, @WildDIC, @AlexanderWillner

if [[ -z "$1" ]]; then
  echo "Syntax: $0 <URL> [<target folder>]" >&2
  exit 1;
fi

command -v jq >/dev/null 2>&1 || {
  echo "Error: needing command 'jq'" >&2
  exit 2;
}

function curl_post_json {
	curl -sH "Content-Type: application/json" -X POST -d "@-" "$@"
}

ALBUM="$(echo $1 | cut -d# -f2)"
BASE_API_URL="https://p23-sharedstreams.icloud.com/${ALBUM}/sharedstreams"

echo "Downloading album '$ALBUM'..."

pushd $2 > /dev/null 2>&1
STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
HOST=$(echo $STREAM | jq '.["X-Apple-MMe-Host"]' | cut -c 2- | rev | cut -c 2- | rev)

echo " - Host: $HOST"
echo " - Stream: $STREAM"

if [ "$HOST" ]; then
    BASE_API_URL="https://$(echo $HOST)/$(echo $1 | cut -d# -f2)/sharedstreams"
    STREAM=$(echo '{"streamCtag":null}' | curl_post_json "$BASE_API_URL/webstream")
fi

CHECKSUMS=$(echo $STREAM | jq -r '.photos[] | [(.derivatives[] | {size: .fileSize | tonumber, value: .checksum})] | max_by(.size | tonumber).value')

echo $STREAM \
| jq -c "{photoGuids: [.photos[].photoGuid]}" \
| curl_post_json "$BASE_API_URL/webasseturls" \
| jq -r '.items | to_entries[] | "https://" + .value.url_location + .value.url_path + "&" + .key' \
| while read URL; do
	for CHECKSUM in $CHECKSUMS; do
		if echo $URL | grep $CHECKSUM > /dev/null; then
			curl -sOJ $URL &
			echo " - Downloading picture $CHECKSUM..."
			break
		fi
	done
done

popd > /dev/null
wait

Hello Alexander, thanks for the script.

There are two issues running in Ubuntu 22.04.

1. line 49, the resulting downloaded file name is too long, and hence no files can be created
2. after some time or some condition (even though the script has been running, downloading part of 1200 video and photos), all subsequent files (at file number 603 and later) will be 12bytes in size. The script will continue to run

I've fixed Issue 1. But for Issue 2 I'm out of my wit.

Any comment?

@charleslcso Hey, thanks - good that this script was useful to you. How did you fix Issue 1? Unfortunately, I'm not using this script anymore, so can't really comment on issue 2.

Hello, my code is not the most efficient and not debugged throughly (about the curl -J option).

I extracted the filename and use it explicitly, and check if it is already downloaded.

                if echo $URL | grep $CHECKSUM > /dev/null; then
                        FILE=$(echo $URL | awk -F'/' '{print $(NF)}' | cut -f1 -d"?")
                        if [[ -f "$2$FILE" ]]; then
                                echo "$FILE exists..."
                        else
                                echo " - Downloading picture $CHECKSUM..."
                                curl $URL -so $2$FILE
                                # or use wget -o "$2$FILE" "$URL"
                        fi
                        break
                fi

Re Issue 2, the reason for the 12Byte file is because Apple server replied a "Unauthorised". This is written into every subsequent files.

Need to find out when would Apple would reply this after N downloads.

Re Issue 2, the reason for the 12Byte file is because Apple server replied a "Unauthorised". This is written into every subsequent files.

Might be some form of rate limiting / timeout ...