Skip to content

Instantly share code, notes, and snippets.

@fczuardi
Last active October 9, 2024 19:58
Show Gist options
  • Save fczuardi/227192cfd566335c31bfe11de59824bf to your computer and use it in GitHub Desktop.
Save fczuardi/227192cfd566335c31bfe11de59824bf to your computer and use it in GitHub Desktop.
Script to reproduce bug #22
-----
Attempt #8
create-bucket, sleep 5, put-versioning, sleep 5, put-object, sleep 5, put-object, sleep 5, purge:
2024/10/09 16:42:14 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "purge" "br-ne1:test-50-1728502909" "-vv" "--retries" "1" "--retries-sleep" "2" "--dump" "headers"]
2024/10/09 16:42:14 DEBUG : Creating backend with remote "br-ne1:test-50-1728502909"
2024/10/09 16:42:14 DEBUG : Using config file from "/home/fcz/.config/rclone/rclone.conf"
2024/10/09 16:42:14 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/10/09 16:42:14 DEBUG : Resolving service "s3" region "br-ne1"
2024/10/09 16:42:14 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/10/09 16:42:14 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:14 DEBUG : HTTP REQUEST (req 0xc000a8c400)
2024/10/09 16:42:14 DEBUG : GET /test-50-1728502909?versioning= HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194214Z
Accept-Encoding: gzip
2024/10/09 16:42:14 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:15 DEBUG : HTTP RESPONSE (req 0xc000a8c400)
2024/10/09 16:42:15 DEBUG : HTTP/1.1 200 OK
Content-Length: 162
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Date: Wed, 09 Oct 2024 19:42:15 GMT
2024/10/09 16:42:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:15 DEBUG : S3 bucket test-50-1728502909: bucket is versioned: true
2024/10/09 16:42:15 DEBUG : Waiting for deletions to finish
2024/10/09 16:42:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : HTTP REQUEST (req 0xc000a8c500)
2024/10/09 16:42:15 DEBUG : GET /test-50-1728502909?delimiter=&max-keys=1000&prefix=&versions= HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194215Z
Accept-Encoding: gzip
2024/10/09 16:42:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:15 DEBUG : HTTP RESPONSE (req 0xc000a8c500)
2024/10/09 16:42:15 DEBUG : HTTP/1.1 200 OK
Content-Length: 1041
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: application/xml
Date: Wed, 09 Oct 2024 19:42:15 GMT
2024/10/09 16:42:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:15 DEBUG : "LICENSE" version false
2024/10/09 16:42:15 DEBUG : LICENSE: Deleting (id "1728502929.40787")
2024/10/09 16:42:15 DEBUG : "LICENSE-v2024-10-09-194203-253" version true
2024/10/09 16:42:15 DEBUG : LICENSE-v2024-10-09-194203-253: Deleting (id "1728502923.25302")
2024/10/09 16:42:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : HTTP REQUEST (req 0xc000a80700)
2024/10/09 16:42:15 DEBUG : DELETE /test-50-1728502909/LICENSE?versionId=1728502923.25302 HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194215Z
Accept-Encoding: gzip
2024/10/09 16:42:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : HTTP REQUEST (req 0xc000a8c900)
2024/10/09 16:42:15 DEBUG : DELETE /test-50-1728502909/LICENSE?versionId=1728502929.40787 HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194215Z
Accept-Encoding: gzip
2024/10/09 16:42:15 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:15 DEBUG : HTTP RESPONSE (req 0xc000a80700)
2024/10/09 16:42:15 DEBUG : HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: text/html; charset=UTF-8
Date: Wed, 09 Oct 2024 19:42:15 GMT
2024/10/09 16:42:15 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:15 INFO : LICENSE-v2024-10-09-194203-253: Deleted
2024/10/09 16:42:16 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:16 DEBUG : HTTP RESPONSE (req 0xc000a8c900)
2024/10/09 16:42:16 DEBUG : HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: text/html; charset=UTF-8
Date: Wed, 09 Oct 2024 19:42:16 GMT
2024/10/09 16:42:16 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:16 INFO : LICENSE: Deleted
2024/10/09 16:42:16 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:16 DEBUG : HTTP REQUEST (req 0xc000209f00)
2024/10/09 16:42:16 DEBUG : DELETE /test-50-1728502909 HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194216Z
Accept-Encoding: gzip
2024/10/09 16:42:16 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:16 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:16 DEBUG : HTTP RESPONSE (req 0xc000209f00)
2024/10/09 16:42:16 DEBUG : HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: text/html; charset=UTF-8
Date: Wed, 09 Oct 2024 19:42:16 GMT
2024/10/09 16:42:16 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:16 INFO : S3 bucket test-50-1728502909: Bucket "test-50-1728502909" deleted
2024/10/09 16:42:16 DEBUG : 8 go routines active
-----
Attempt #9
create-bucket, sleep 5, put-versioning, sleep 5, put-object, sleep 5, put-object, sleep 5, purge:
2024/10/09 16:42:42 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "purge" "br-ne1:test-50-1728502936" "-vv" "--retries" "1" "--retries-sleep" "2" "--dump" "headers"]
2024/10/09 16:42:42 DEBUG : Creating backend with remote "br-ne1:test-50-1728502936"
2024/10/09 16:42:42 DEBUG : Using config file from "/home/fcz/.config/rclone/rclone.conf"
2024/10/09 16:42:42 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/10/09 16:42:42 DEBUG : Resolving service "s3" region "br-ne1"
2024/10/09 16:42:42 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/10/09 16:42:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:42 DEBUG : HTTP REQUEST (req 0xc000839300)
2024/10/09 16:42:42 DEBUG : GET /test-50-1728502936?versioning= HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194242Z
Accept-Encoding: gzip
2024/10/09 16:42:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:42 DEBUG : HTTP RESPONSE (req 0xc000839300)
2024/10/09 16:42:42 DEBUG : HTTP/1.1 200 OK
Content-Length: 162
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Date: Wed, 09 Oct 2024 19:42:42 GMT
2024/10/09 16:42:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:42 DEBUG : S3 bucket test-50-1728502936: bucket is versioned: true
2024/10/09 16:42:42 DEBUG : Waiting for deletions to finish
2024/10/09 16:42:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:42 DEBUG : HTTP REQUEST (req 0xc000279000)
2024/10/09 16:42:42 DEBUG : GET /test-50-1728502936?delimiter=&max-keys=1000&prefix=&versions= HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194242Z
Accept-Encoding: gzip
2024/10/09 16:42:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:43 DEBUG : HTTP RESPONSE (req 0xc000279000)
2024/10/09 16:42:43 DEBUG : HTTP/1.1 200 OK
Content-Length: 1041
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: application/xml
Date: Wed, 09 Oct 2024 19:42:43 GMT
2024/10/09 16:42:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:43 DEBUG : "LICENSE" version false
2024/10/09 16:42:43 DEBUG : LICENSE: Deleting (id "1728502956.94820")
2024/10/09 16:42:43 DEBUG : "LICENSE-v2024-10-09-194230-856" version true
2024/10/09 16:42:43 DEBUG : LICENSE-v2024-10-09-194230-856: Deleting (id "1728502950.85635")
2024/10/09 16:42:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:43 DEBUG : HTTP REQUEST (req 0xc000500e00)
2024/10/09 16:42:43 DEBUG : DELETE /test-50-1728502936/LICENSE?versionId=1728502956.94820 HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194243Z
Accept-Encoding: gzip
2024/10/09 16:42:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:43 DEBUG : HTTP REQUEST (req 0xc000839700)
2024/10/09 16:42:43 DEBUG : DELETE /test-50-1728502936/LICENSE?versionId=1728502950.85635 HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194243Z
Accept-Encoding: gzip
2024/10/09 16:42:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:43 DEBUG : HTTP RESPONSE (req 0xc000500e00)
2024/10/09 16:42:43 DEBUG : HTTP/1.1 204 Created
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: text/html; charset=UTF-8
Date: Wed, 09 Oct 2024 19:42:43 GMT
Etag: "9bf483dc6de8399d4a0ad4614958e81d"
Last-Modified: Wed, 09 Oct 2024 19:42:44 GMT
2024/10/09 16:42:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:43 INFO : LICENSE: Deleted
2024/10/09 16:42:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:43 DEBUG : HTTP RESPONSE (req 0xc000839700)
2024/10/09 16:42:43 DEBUG : HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: text/html; charset=UTF-8
Date: Wed, 09 Oct 2024 19:42:43 GMT
2024/10/09 16:42:43 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:43 INFO : LICENSE-v2024-10-09-194230-856: Deleted
2024/10/09 16:42:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:43 DEBUG : HTTP REQUEST (req 0xc0005da400)
2024/10/09 16:42:43 DEBUG : DELETE /test-50-1728502936 HTTP/1.1
Host: br-ne1.magaluobjects.com
User-Agent: rclone/v1.64.0
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20241009T194243Z
Accept-Encoding: gzip
2024/10/09 16:42:43 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/10/09 16:42:44 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:44 DEBUG : HTTP RESPONSE (req 0xc0005da400)
2024/10/09 16:42:44 DEBUG : HTTP/1.1 409 Conflict
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Connection: keep-alive
Connection: keep-alive
Content-Type: application/xml
Date: Wed, 09 Oct 2024 19:42:43 GMT
2024/10/09 16:42:44 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/10/09 16:42:44 ERROR : Attempt 1/1 failed with 1 errors and: BucketNotEmpty: The bucket you tried to delete is not empty
status code: 409, request id: , host id:
2024/10/09 16:42:46 DEBUG : 8 go routines active
2024/10/09 16:42:46 Failed to purge: BucketNotEmpty: The bucket you tried to delete is not empty
status code: 409, request id: , host id:
#!/bin/bash
[ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ] && {
echo "usage: script.sh <profile> <bucket prefix> <file to upload> <sleep time before purge>"
echo "example: ./make-undeletable-bucket.sh br-ne1 test-50 LICENSE 0"
exit 1
}
count=1
RCLONE_PURGE_OPTIONS="-vv --retries 1 --retries-sleep 2 --dump headers"
while true; do
PROFILE=$1
BUCKET=$2-$(date +%s)
FILE=$3
SLEEP=$4
echo "Attempt #$((count++))"
printf "create-bucket"
aws s3api create-bucket --profile $PROFILE --bucket "$BUCKET" > /dev/null || break
printf ", sleep $SLEEP"
sleep $SLEEP
printf ", put-versioning"
aws s3api --profile $PROFILE put-bucket-versioning --bucket $BUCKET --versioning-configuration Status=Enabled > /dev/null || break
printf ", sleep $SLEEP"
sleep $SLEEP
printf ", put-object"
aws s3api put-object --profile $PROFILE --bucket $BUCKET --key $FILE --body $FILE > /dev/null || break
printf ", sleep $SLEEP"
sleep $SLEEP
printf ", put-object"
aws s3api put-object --profile $PROFILE --bucket $BUCKET --key $FILE --body $FILE > /dev/null || break
printf ", sleep $SLEEP"
sleep $SLEEP
echo ", purge:"
rclone purge $PROFILE:$BUCKET $RCLONE_PURGE_OPTIONS || break
echo "-----";
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment