backend notes

backend-notes.md

Tips for the creation of new backends!

General tips:

1. First read some documentation that can help in this task:

• https://docs.ibis-project.org/contributing.html
• https://docs.ibis-project.org/backends.html

2. Run make develop locally that will install pre-commit hook and prepare your environment for development.
3. Maybe you want to add the new backend as pytest marker: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/setup.cfg#L34
4. If the tests for your backend is not available for Windows CI, add a pytest expression marker that will skip tests for the backend here: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ci/azure/windows.yml#L130
5. Also, if the tests for your backend is not available for Linux CI, add a pytest expression marker that will skip tests for the backend into PYTEST_MARK_EXPRESSION environment variable : https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ci/azure/linux.yml#L23
6. Add a function for the new backend for data loading, use this as example: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ci/datamgr.py#L236
7. Add the new backend name into the data load scripts: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ci/load-data.sh#L5
8. Create the SQL file for tables creation, you can use postgresql.sql as base: ./ci/schema
9. If the new database server is available for Linux, add the image and configurations for the new database here: ./ci/docker-compose.yml 9.1. Add the new database in the waiter call: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ci/docker-compose.yml#L93
10. If the new database server is available for windows, add the script installation into Azure Pipelines receipt, you can follow this as base: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ci/azure/windows.yml#L86
11. Add the dependencies into the "requirements-*-dev.yml": ./ci
12. Add the dependencies into setup.py, check how this file configures that for the other backend: ./setup.py
13. If the new backend is a "String generating backend" create a folder for the new backend at ./ibis 13.1. You can use another backend as base, such as ./ibis/omniscidb
14. If the new backend is a "Expression generating backends" create a folder fot the new backend at ./ibis/sql.
15. Import your new backend into ./init..py, you can follow this example: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ibis/__init__.py#L60
16. Tests 16.1. General tests are stored at ./ibis/tests/all/ 16.2. Create a new backend for the general tests on ./ibis/tests/backend.py, you can follow this example: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ibis/tests/backends.py#L286 16.3. Generally, tests for unsupported operations will be skipt. 16.4. If some test is failling for the new backend, make sure that operations is marked as unsupported. There is an example here how to force that: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ibis/omniscidb/operations.py#L913 16.5. If you want to add a specific test for the new backend, you can add it at ./ibis/MY_NEW_BACKEND/tests. You can check an example here: https://github.com/ibis-project/ibis/blob/99a2f2e4685a137f64f8242bee0ec44695ede34f/ibis/omniscidb/tests/test_client.py

Tips for mssql backend:

1. It would be good to install it into azure-pipelines for windows

• maybe it could help: https://chocolatey.org/packages/sql-server-2017