ivangeorgiev

data = [
    ('1990-05-03', 29, True),
    ('1994-09-23', 25, False)
]

df = spark.createDataFrame(data, ['dob', 'age', 'is_fan'])
df.printSchema()

ivangeorgiev

# https://docs.databricks.com/dev-tools/api/latest/secrets.html
# https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-6

if ( ! $DBConfig ) {
    $DBConfig = @{
        BaseUri = "https://westeurope.azuredatabricks.net/api"
        Token = "XXXXXXXXXXXXXXXXXXXXXXXXX"
    }
}

ivangeorgiev

# https://docs.microsoft.com/en-us/previous-versions/powershell/module/Microsoft.PowerShell.Utility/Send-MailMessage?view=powershell-5.0

$smtp = "smtp.office365.com"


$username = "username here ..."
$passwordText = "password here ..."

$from = $username
$to = $username

ivangeorgiev

function Templetize-ArmTemplateParams {
    param(
        $InputParamsFile
    )

    $json = Get-Content $InputParamsFile | Out-String | ConvertFrom-Json
    foreach($param in $json.parameters.PSObject.Properties)
    {

ivangeorgiev

Problem

When storing JWT, e.g. returned from Azure Active Directory endpoints, into cookies, web server (Tomcat) throws an exception or warning regarding limit of cookie size.

Discussion

Most browsers limit the size of a cookie to 4096, including name and =. There are also limits on the number of cookies per domain. JWT is not limited in size.

• A good information regarding limtations on cookies: http://browsercookielimits.squawky.net/

ivangeorgiev

Build and test Python apps

ivangeorgiev

• Yaml Schema Reference
• Pipeline variables

ivangeorgiev

Application ID Servie Principal Secret Resource Group Name Databricks Workspace Name Subscription ID TenantID

ivangeorgiev

import re
import os
import glob
import collections
import contextlib

PatchEntry = collections.namedtuple('PatchEntry', 'version,path')

patch_key = lambda x: x.version

ivangeorgiev

import itertools

class Pipe:
    def __init__(self, seq):
        self._seq = seq

    def __iter__(self):
        return iter(self._seq)
    
    def map(self, func):