Unit testing with Databricks

test.md

Unit Testing with Databricks

pytest-mock

sys.dont_write_bytecode = True # or set init script to allow writing to repo for pytest.

dbutils.library.restartPython() # to clear the interpreter before a test.

Find the repo root

notebook_path = dbutils.notebook.entry_point.getDbutils().notebook().getContext().notebookPath().get()
repo_root = os.path.dirname(os.path.dirname(notebook_path))
os.chdir(f'/Workspace/{repo_root}')

retcode = pytest.main([".", "-p", "no:cacheprovider"])

# Fail the cell execution if we have any test failures.
assert retcode == 0, 'The pytest invocation failed. See the log above for details.'

https://github.com/databricks/notebook-best-practices/blob/main/notebooks/run_unit_tests.py

Writing test outputs to the FileStore

import pytest

retcode = pytest.main(['-k', 'mycode_test',
                       '-o', 'cache_dir=/dbfs/FileStore/',
                       '--junitxml', '/dbfs/FileStore/pytestreport.xml',
                       '-v'
                      ]
                     )