azure curl from http://stackoverflow.com/questions/20103258/accessing-azure-blob-storage-using-bash-curl#30608623

gistfile1.txt

#!/bin/bash

# List the blobs in an Azure storage container.

echo "usage: ${0##*/} <container-name> [blob-name]"

storage_account="$AZURE_STORAGE_ACCOUNT"
container_name="$1"
access_key="$AZURE_STORAGE_KEY"

blob_store_url="blob.core.windows.net"
authorization="SharedKey"

request_method="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2011-08-18"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"

# Build the signature string
canonicalized_headers="${x_ms_date_h}\n${x_ms_version_h}"
canonicalized_resource="/${storage_account}/${container_name}\ncomp:list\nrestype:container"
[[ "" != "$2" ]] && canonicalized_resource="/${storage_account}/${container_name}/$2"

string_to_sign="${request_method}\n\n\n\n\n\n\n\n\n\n\n\n${canonicalized_headers}\n${canonicalized_resource}"

# Decode the Base64 encoded access key, convert to Hex.
decoded_hex_key="$(echo -n $access_key | base64 -d -w0 | xxd -p -c256)"

# Create the HMAC signature for the Authorization header
signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary | base64 -w0)

authorization_header="Authorization: $authorization $storage_account:$signature"

URL="https://${storage_account}.${blob_store_url}/${container_name}?restype=container&comp=list"

[[ "" != "$2" ]] && URL="https://${storage_account}.${blob_store_url}/${container_name}/$2"

curl -v \
  -H "$x_ms_date_h" \
  -H "$x_ms_version_h" \
  -H "$authorization_header" \
  "$URL"

Thanks for this. Really useful example.

It was much useful

Very helpful, thanks a lot for this. It works great for list and downloads with GET.

I tried to modify it for uploads with PUT but it fails to authenticate.

I use the same key, sig, and string_to_sign, authorization header, etc. as in the working GETs but the PUTs give
< HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

I checked the azure docs to make sure i'm following the API for PUTs. here's my PUT version. Any ideas very appreciated.

#!/bin/bash

echo "usage: ${0##*/} <container-name> [blob-name]"

storage_account="ab2022storage1"
container_name="$1"
access_key="$MY_ACCESS_KEY"
blob_store_url="blob.core.windows.net"
authorization="SharedKey"
request_method="PUT"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2015-02-21"

# HTTP Request headers
x_ms_date_h="x-ms-date: $request_date"
x_ms_version_h="x-ms-version: $storage_service_version"
x_ms_blobtype="x-ms-blob-type: BlockBlob"

# Build the signature string
canonicalized_headers="${x_ms_date_h}\n${x_ms_version_h}"
canonicalized_resource="/${storage_account}/${container_name}\ncomp:list\nrestype:container"
[[ "" != "$2" ]] && canonicalized_resource="/${storage_account}/${container_name}\nrestype:container\ntimeout=30"

string_to_sign="${request_method}\n\n\n\n\n\n\n\n\n\n\n\n${canonicalized_headers}\n${canonicalized_resource}"

# Decode the Base64 encoded access key, convert to Hex.
decoded_hex_key="$(echo -n $access_key | base64 -d -w0 | xxd -p -c256)"

# Create the HMAC signature for the Authorization header
signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary | base64 -w0)

authorization_header="Authorization: $authorization $storage_account:$signature"

URL="https://${storage_account}.${blob_store_url}/${container_name}?restype=container&comp=list"
[[ "" != "$2" ]] && URL="https://${storage_account}.${blob_store_url}/${container_name}\nrestype:container\ntimeout=30"

curl -v -X PUT --data-binary @"$2" \
  -H "Content-Type: text/plain; charset=UTF-8" \
  -H "$x_ms_date_h" \
  -H "$x_ms_version_h" \
  -H "$x_ms_blobtype" \
  -H "$authorization_header" \
  "$URL"

Hi @ab2022 here's a working example. One good troubleshooting step is that Azure will return the string it's signing for comparison, and you can compare that to your string_to_sign by echoing that out. In mine I was having problems with \n so I just included explicit newlines.

#!/usr/bin/env bash

# usage: ./azure-upload-blob.sh <account-name> <container-name> <blob-name> <file-to-upload>
# e.g.:  ./azure-upload-blob.sh myaccount mycontainer sample.txt sample.txt

storage_account="$1"
container_name="$2"
blob_name="$3"
file_path="$4"

if [[ -z "$storage_account" || -z "$container_name" || -z "$blob_name" || -z "$file_path" ]]; then
  echo "Usage: ${0##*/} <account-name> <container-name> <blob-name> <file-to-upload>"
  exit 1
fi

access_key="${BLOB_ACCESS_KEY}"   # Ensure $MY_ACCESS_KEY is set in your env
blob_store_url="blob.core.windows.net"
authorization="SharedKey"
request_method="PUT"
request_date="$(TZ=GMT date '+%a, %d %b %Y %H:%M:%S GMT')"
storage_service_version="2021-04-10"

if [[ -z "$access_key" ]]; then
  echo "Missing BLOB_ACCESS_KEY environment variable."
  exit 1
fi

###############################################################################
# 1) Gather info for signing
###############################################################################
# We need the size in bytes for Content-Length (required for PUT w/ body).
file_size="$(stat -c%s "$file_path" 2>/dev/null || stat -f%z "$file_path")"

# Headers
x_ms_date_h="x-ms-date: ${request_date}"
x_ms_version_h="x-ms-version: ${storage_service_version}"
x_ms_blobtype_h="x-ms-blob-type: BlockBlob"
content_length_h="Content-Length: ${file_size}"

# We’ll include content-type, but note that for Azure signing
# if you include it in the request, you also put it in the string-to-sign
content_type="text/plain; charset=UTF-8"
content_type_h="Content-Type: ${content_type}"

###############################################################################
# 2) Build the canonicalized headers
###############################################################################
# For the string-to-sign, all x-ms-* headers must be lowercased
canonicalized_headers="x-ms-blob-type:BlockBlob
x-ms-date:${request_date}
x-ms-version:${storage_service_version}"

# The "canonicalized resource" for blob uploads is:
#   /{account-name}/{container-name}/{blob-name}
#   No extra comp= stuff unless you’re doing block commits, snapshots, etc.
canonicalized_resource="/${storage_account}/${container_name}/${blob_name}"

###############################################################################
# 3) Construct the string to sign
###############################################################################
# The official docs show the format for the “Blob Service” string-to-sign.
# https://learn.microsoft.com/en-us/rest/api/storagerp/blobservices/authentication-for-the-azure-storage-services
# The general pattern for PUT with these basic headers is:
#
#   VERB + "\n" +
#   Content-Encoding + "\n" +
#   Content-Language + "\n" +
#   Content-Length + "\n" +    (if this is an empty string, use "0" instead)
#   Content-MD5 + "\n" +
#   Content-Type + "\n" +
#   Date + "\n" +
#   If-Modified-Since + "\n" +
#   If-Match + "\n" +
#   If-None-Match + "\n" +
#   If-Unmodified-Since + "\n" +
#   Range + "\n" +
#   CanonicalizedHeaders + "\n" +
#   CanonicalizedResource
#
# We’ll keep empty strings for all except content-length, content-type, and the x-ms-* headers.
# This was giving me issues using `\n` so I just broke it out in the string. On Windows be careful
# with the line endings, CRLF vs LF.
string_to_sign="${request_method}


${file_size}

${content_type}






${canonicalized_headers}
${canonicalized_resource}"

###############################################################################
# 4) Create the HMAC SHA256 signature
###############################################################################
decoded_hex_key="$(echo -n "$access_key" | base64 -d | xxd -p -c256)"
signature="$(printf "%s" "${string_to_sign}" \
  | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${decoded_hex_key}" -binary \
  | base64 -w0)"

authorization_header="Authorization: ${authorization} ${storage_account}:${signature}"

###############################################################################
# 5) Make the HTTP request to upload
###############################################################################
url="https://${storage_account}.${blob_store_url}/${container_name}/${blob_name}"

# Use --data-binary to preserve file contents as-is
curl -v -X PUT \
  --data-binary "@${file_path}" \
  -H "${content_type_h}" \
  -H "${content_length_h}" \
  -H "${x_ms_blobtype_h}" \
  -H "${x_ms_date_h}" \
  -H "${x_ms_version_h}" \
  -H "${authorization_header}" \
  "${url}"

@ChrisRomp thanks! works great!... good tip about comparing against what azure returns. I had the canonicalized stuff wrong.