Skip to content

Instantly share code, notes, and snippets.

@mmckechney
Last active June 13, 2019 17:28
Show Gist options
  • Save mmckechney/c8dfcf4f337aa4cbe48a44591c89dfa0 to your computer and use it in GitHub Desktop.
Save mmckechney/c8dfcf4f337aa4cbe48a44591c89dfa0 to your computer and use it in GitHub Desktop.
PowerShell to create ADLS Gen2 filesystem, path and file via REST
$AadTenant = "" # <-- AAD tenant ID
$AadAppId = "" # <-- App Id of the identity to use
$AadAppKey = "" # <-- Secret key of this identity
$AdlsAccountName = "" # <-- name of your ASLD Gen2 account
$FileSystemName = "" # <-- name of the file system to create
$DirPath = "" # <-- Directory path to create
$FileName = "" # <-- File name to create on ADLS
$FilePath = "" # <-- local path to file
#Get OAuth2 Access Token from Azure AD
$body = @{
"grant_type" = "client_credentials"
"client_id" = "$AadAppId"
"client_secret" = "$AadAppKey"
"resource" = "https://storage.azure.com"
"scope" = "https://storage.azure.com/.default"
}
$authResult = Invoke-RestMethod -Uri "https://login.microsoftonline.com/$AadTenant/oauth2/token" -Body $body -Method Post
#Make call to ADLS Gen2 to create File System
$headers = @{
"x-ms-version" = "2018-11-09"
"Authorization" = "Bearer $($authResult.access_token)"
}
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)?resource=filesystem"
Invoke-RestMethod -Uri $url -Headers $headers -Method Put
#Make call to ADLS Gen2 to create directory path
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)?resource=directory"
Invoke-RestMethod -Uri $url -Headers $headers -Method Put
#Make call to ADLS Gen2 to create file
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)/$($FileName)?resource=file"
Invoke-RestMethod -Uri $url -Headers $headers -Method PUT
#File step 1:Make call to upload file contents
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)/$($FileName)?action=append&timeout=1000&position=0";
Invoke-RestMethod -Uri $url -Headers $headers -Method PATCH -InFile $FilePath
#File step 2: Make call to commit contents (flush buffer)
$byteContent = [System.IO.File]::ReadAllBytes($FilePath)
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)/$($FileName)?action=flush&timeout=1000&position=$($byteContent.Length)";
Invoke-RestMethod -Uri $url -Headers $headers -Method PATCH
@mmckechney
Copy link
Author

mmckechney commented May 24, 2019

Set-up

In order to use the script, you will need an account to authenticate against. This can be an app identity or service account. The easiest way to do this is to create an new Azure AD App and generate the client secret (there are some good instructions here).

Important You will need to make the App account a contributor to the ADLS

Once you have created it, you can use the Application ID and secret for the values of $AadAppId and $AadAppKey.
In addition, you will need value for the following variables:

$AadTenant - the GUID value of your AAD tenant. If you don't know this, you can easily get it via the Azure Portal. Select "Azure Active Directory" --> Properties -- it is the Directory ID value
$AdlsAccountName - name of your ASLD Gen2 account
$FileSystemName - name of the file system hierarchy to create

How it works

There are two Invoke-RestMethod calls in the script. The first one calls out to your AAD tenant with the AppId and secret to generate and return an OAuth2 bearer token. This token is then used as authentication in the second call out to your ADLS Gen2 account to create the file system hierarchy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment