Skip to content

Instantly share code, notes, and snippets.

@zeddee

zeddee/azure-grok.rst

Last active March 21, 2020 23:58
Show Gist options
  • Save zeddee/1299dfa1b1969766ef5eb2cdf12e9232 to your computer and use it in GitHub Desktop.
Save zeddee/1299dfa1b1969766ef5eb2cdf12e9232 to your computer and use it in GitHub Desktop.
Download ZIP
added suffix to sectnum directive
Raw
azure-grok.rst

Azure Grok

1.   Getting started

Set up credentials in environment variables:

AZURE_STORAGE_ACCOUNT=<storage-account-name>
AZURE_STORAGE_KEY=<storage-account-key>
AZURE_STORAGE_CONNECTION_STRING=<connection-string>
<storage-account-name>:

Get by running:

az storage account list
(Optional) <storage-account-key>:
  • If using key-based authentication. Deprecated?
  • Not required if running az with --auth-mode login
(Optional) <connection-string>:
  • ... not sure what this does yet.

2.   Authenticate

Run:

az login

3.   Get list of Azure accounts

Run:

az account list

Output:

[
  {
    "cloudName": "AzureCloud",
    "id": "4809d4ef-e5af-45b3-af5e-b67c700eb7de",
    "isDefault": false,
    "name": "subscription 1",
    "state": "Enabled",
    "tenantId": "de489e9a-2c2c-4c45-8945-6cd059d4d655",
    "user": {
      "name": "[email protected]",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "de489e9a-2c2c-4c45-8945-6cd059d4d655",
    "isDefault": true,
    "name": "subscription 2",
    "state": "Enabled",
    "tenantId": "de489e9a-2c2c-4c45-8945-6cd059d4d655",
    "user": {
      "name": "[email protected]",
      "type": "user"
    }
  }
]

4.   Set active subscription

az account set --subscription "subscription 2"

5.   Show resource groups

az group list

az group show --name <resource-group-name>

Output:

{
  "id": "/subscriptions/38146078-557a-454f-ae56-1302b0b67582/resourceGroups/<resource-group-name>",
  "location": "eastasia",
  "managedBy": null,
  "name": "<resource-group-name>",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

6.   List storage containers

Run:

az storage container list --auth-mode login --account-name $AZURE_STORAGE_ACCOUNT

Output:

[
  {
    "metadata": null,
    "name": "$web",
    "properties": {
      "etag": "\"0x8D79D41E7F0BEF7\"",
      "hasImmutabilityPolicy": "false",
      "hasLegalHold": "false",
      "lastModified": "2020-01-20T00:44:28+00:00",
      "lease": {
        "duration": null,
        "state": null,
        "status": null
      },
      "leaseDuration": null,
      "leaseState": "available",
      "leaseStatus": "unlocked",
      "publicAccess": null
    }
  }
]

7.   --auth-mode login bug

There appears to be a bug when attempting to access resources using --auth-mode login:

az storage blob list --auth-mode login --container-name '$web' --account-name $AZURE_STORAGE_ACCOUNT

# Output:
You do not have the required permissions needed to perform this operation.
Depending on your operation, you may need to be assigned one of the following roles:
    "Storage Blob Data Contributor"
    "Storage Blob Data Reader"
    "Storage Queue Data Contributor"
    "Storage Queue Data Reader"

If you want to use the old authentication method and allow querying for the right account key, please use the "--auth-mode" parameter and "key" value.

If you encounter this message, run your command with --auth-mode key instead:

az storage blob list --auth-mode key --container-name '$web' --account-name $AZURE_STORAGE_ACCOUNT --account-key $AZURE_STORAGE_KEY --connection-string $AZURE_CONNECTION_STRING

# Output:
[]

8.   Nested storage containers

You can't nest storage containers. Containers have a flat heirarchy:

https://stackoverflow.com/questions/3183857/how-to-create-a-sub-container-in-azure-storage-location

9.   AccountSASPermissions

from azure.storage.blob import ResourceTypes, AccountSASPermissions, generate_account_sas

sas_token = generate_account_sas(
    blob_service_client.account_name,
    account_key=blob_service_client.credential.account_key,
    resource_types=ResourceTypes(object=True),
    permission=AccountSasPermissions(read=True),
    expiry=datetime.utcnow() + timedelta(hours=1)
)

# ResourceTypes(self, service=False, container=False, object=False)
#
# AccountSASPermissions(self, read=False, write=False, delete=False, list=False,  # pylint: disable=redefined-builtin
#             add=False, create=False, update=False, process=False)
#
# generate_account_sas(
#     account_name,  # type: str
#     account_key,  # type: str
#     resource_types,  # type: Union[ResourceTypes, str]
#     permission,  # type: Union[AccountSasPermissions, str]
#     expiry,  # type: Optional[Union[datetime, str]]
#     start=None,  # type: Optional[Union[datetime, str]]
#     ip=None,  # type: Optional[str]
#     **kwargs # type: Any
# ):  # type: (...) -> str(SAS token)

10.   Difference between BlobClient and BlobServiceClient

BlobClient is used to interact with specific blobs:

BlobClient(account_url, container_name, blob_name, snapshot=None, credential=None, **kwargs)

Whereas the BlobServiceClient is used to interact with the Blob service at the storage account level:

BlobServiceClient(account_url, credential=None, **kwargs)

11.   Restrict access to on-premises network

It appears that there isn't a way to quickly configure access to the static website endpoint for blobs so that we restrict access to only, let's say, signed in Azure AD users.

What the Azure storage blob docs do recommend is:

  1. Finding the public IP range for the on-premises network

  2. And then set the vnet/firewall for the storage account to DENY ALL and ALLOW <this_public_ip_range> to allow only devices on the ip ranges your devices reside on to access the blob.

    brute force, but i suppose that works.

Source: https://docs.microsoft.com/en-gb/azure/storage/common/storage-network-security#configuring-access-from-on-premises-networks

12.   Deleting Blobs on Azure Storage Blob using Python SDK

Have been trouble using the Python SDK to delete blobs. When I call the BlobServiceClient.client.delete_blobs(*blob) method on a given blob:

  • Works with no issue when using an Access Key connection string

  • Fails (silently, ie. it doesn't throw a panic) with the following Python error:

    'NoneType' object has no attribute 'on_request'

Consulted a colleague, and he suggested calling the REST API instead, which I'm attempting right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment