Last active
June 13, 2019 17:28
-
-
Save mmckechney/c8dfcf4f337aa4cbe48a44591c89dfa0 to your computer and use it in GitHub Desktop.
PowerShell to create ADLS Gen2 filesystem, path and file via REST
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$AadTenant = "" # <-- AAD tenant ID | |
$AadAppId = "" # <-- App Id of the identity to use | |
$AadAppKey = "" # <-- Secret key of this identity | |
$AdlsAccountName = "" # <-- name of your ASLD Gen2 account | |
$FileSystemName = "" # <-- name of the file system to create | |
$DirPath = "" # <-- Directory path to create | |
$FileName = "" # <-- File name to create on ADLS | |
$FilePath = "" # <-- local path to file | |
#Get OAuth2 Access Token from Azure AD | |
$body = @{ | |
"grant_type" = "client_credentials" | |
"client_id" = "$AadAppId" | |
"client_secret" = "$AadAppKey" | |
"resource" = "https://storage.azure.com" | |
"scope" = "https://storage.azure.com/.default" | |
} | |
$authResult = Invoke-RestMethod -Uri "https://login.microsoftonline.com/$AadTenant/oauth2/token" -Body $body -Method Post | |
#Make call to ADLS Gen2 to create File System | |
$headers = @{ | |
"x-ms-version" = "2018-11-09" | |
"Authorization" = "Bearer $($authResult.access_token)" | |
} | |
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)?resource=filesystem" | |
Invoke-RestMethod -Uri $url -Headers $headers -Method Put | |
#Make call to ADLS Gen2 to create directory path | |
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)?resource=directory" | |
Invoke-RestMethod -Uri $url -Headers $headers -Method Put | |
#Make call to ADLS Gen2 to create file | |
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)/$($FileName)?resource=file" | |
Invoke-RestMethod -Uri $url -Headers $headers -Method PUT | |
#File step 1:Make call to upload file contents | |
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)/$($FileName)?action=append&timeout=1000&position=0"; | |
Invoke-RestMethod -Uri $url -Headers $headers -Method PATCH -InFile $FilePath | |
#File step 2: Make call to commit contents (flush buffer) | |
$byteContent = [System.IO.File]::ReadAllBytes($FilePath) | |
$url = "https://$AdlsAccountName.dfs.core.windows.net/$($FileSystemName)/$($DirPath)/$($FileName)?action=flush&timeout=1000&position=$($byteContent.Length)"; | |
Invoke-RestMethod -Uri $url -Headers $headers -Method PATCH |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Set-up
In order to use the script, you will need an account to authenticate against. This can be an app identity or service account. The easiest way to do this is to create an new Azure AD App and generate the client secret (there are some good instructions here).
Important You will need to make the App account a contributor to the ADLS
Once you have created it, you can use the Application ID and secret for the values of
$AadAppId
and$AadAppKey
.In addition, you will need value for the following variables:
$AadTenant
- the GUID value of your AAD tenant. If you don't know this, you can easily get it via the Azure Portal. Select "Azure Active Directory" --> Properties -- it is the Directory ID value$AdlsAccountName
- name of your ASLD Gen2 account$FileSystemName
- name of the file system hierarchy to createHow it works
There are two
Invoke-RestMethod
calls in the script. The first one calls out to your AAD tenant with the AppId and secret to generate and return an OAuth2 bearer token. This token is then used as authentication in the second call out to your ADLS Gen2 account to create the file system hierarchy