asears

Table of Contents

• set -e, -u, -x, -o pipefail
• set -e
• set -x
• set -u
• set -o pipefail
• Setting IFS
• Original Reference

set -e, -u, -x, -o pipefail

asears

rustc --version cargo --version

cargo install cargo-quickinstall

cargo quickinstall gitoxide

The default installation, 'max'

cargo install gitoxide

asears

@echo off
REM spleeter to convert to wav with 4 stems, convert to mp3 with minimal loss.
spleeter separate -p spleeter:4stems -o output %1
SET _file=%1
SET _folder=%_file:.mp3=%
SET _folder=%_folder:"=%
ECHO %_folder%
ffmpeg -vn -ar 44100 -ac 2 -b:a 192k "output\%_folder%\vocals.mp3" -i "output\%_folder%\vocals.wav" 
ffmpeg -vn -ar 44100 -ac 2 -b:a 192k "output\%_folder%\bass.mp3" -i "output\%_folder%\bass.wav"
ffmpeg -vn -ar 44100 -ac 2 -b:a 192k "output\%_folder%\drums.mp3" -i "output\%_folder%\drums.wav"

asears

Unit Testing with Databricks

pytest-mock

sys.dont_write_bytecode = True # or set init script to allow writing to repo for pytest.

dbutils.library.restartPython() # to clear the interpreter before a test.

asears

https://www.red-gate.com/simple-talk/blogs/sql-naming-conventions/

It helps greatly to start with a dictionary of the correct nouns and verbs associated with the application domain and use that.

If it proves inadequate, then the team can build on it.

If a data model has been created as part of the design phase, this dictionary should be an end-product of this work.

Don't tibble, no tbl_ or rpt_

asears

SELECT name
  FROM sys.objects
    INNER JOIN
      (
      VALUES ('ADD'), ('EXTERNAL'), ('PROCEDURE'), ('ALL'), ('FETCH'),
        ('PUBLIC'), ('ALTER'), ('FILE'), ('RAISERROR'), ('AND'),
        ('FILLFACTOR'), ('READ'), ('ANY'), ('FOR'), ('READTEXT'), ('AS'),
        ('FOREIGN'), ('RECONFIGURE'), ('ASC'), ('FREETEXT'), ('REFERENCES'),
        ('AUTHORIZATION'), ('FREETEXTTABLE'), ('REPLICATION'), ('BACKUP'),
        ('FROM'), ('RESTORE'), ('BEGIN'), ('FULL'), ('RESTRICT'), ('BETWEEN'),

asears

# Get-ChildItem -Recurse -Filter "*.bicep" | foreach { Test-BicepFile -Path $_.fullname }

# https://github.com/Azure/bicep/issues/5828

$biceps = Get-ChildItem -Path . -Include "*.bicep" -r

$errors = 0
$numberOfFiles = $biceps.Length

foreach ($file in $biceps) {

asears

# Global user
git config --global user.name "Firstname lastname"
git config --global user.email [email protected]
# Local user
git config user.name=<user name>
git config user.email=<email>

git config credential.helper store

asears

Azure Templates and Patterns

Data Product Streaming Template

https://github.com/Azure/data-product-streaming

Data Product Batch

https://github.com/Azure/data-product-batch

asears

Pytest Resources

Pytest—Paramaterize tests with external data

https://remusao.github.io/posts/pytest-param.html

API Reference

https://docs.pytest.org/en/6.2.x/reference.html#request