aurweb - containerized dev/test environment setup

setup.md

aurweb - containerized dev/test environment setup

This guide aims to explain how to set up an aurweb environment for local development and testing.
We'll make use of the containerized (docker) version of aurweb.

Prerequisites

We need docker-compose to be able to build our set of containers.
For development and testing we need python-poetry which creates a virtual environment
and installs all required python packages for us.
We'll also need mariadb-libs for our python mysqlclient package.

# pacman -S --needed docker docker-compose python-poetry mariadb-libs asciidoc openssh

Start the docker service with # systemctl start docker

Building the environment

Let's create our set of containers and run them.

1. Clone the auweb project
   (if you plan to contribute, create a fork and use the ssh URI when cloning)

   $ git clone https://gitlab.archlinux.org/archlinux/aurweb.git
   $ cd aurweb
   

2. Build image
   Let's download the archlinux image and build our aurweb image. docker compose will do all this for us:

   # docker compose build
   

3. Create and run our containers
   Now that we have our image, let's create and run our containers

   # docker compose up
   

   You should have aurweb running at https://localhost:8444/
   In case it exited with exit code 2 for the ca container, rerun docker compose up

4. Feed it with dummy data
   Alright, we got it running. Let's feed it with some dummy data.
   Open another shell and login to the mariadb container.

   # docker compose exec mariadb /bin/bash
   # pacman -S --noconfirm words fortune-mod
   # poetry run schema/gendummydata.py dummy_data.sql
   # mysql -uaur -paur aurweb < dummy_data.sql
   # exit
   

Nice, now we got our environment set up.
Code is hot reloaded when a file is being saved.
You can connect to the DB instance on port 13306 with user root and password aur.
Stop the cluster with CTRL+C.

Developing & Testing

Ok, now we got aurweb running locally and want to start development.
We can make use of container that runs all tests for us with # docker compose run test.
However, during development we'd want to run our tests individually, especially when writing tests.
For that, we'll need to do a couple of additional things.
We should set up a virtual environment and install the required python packages.
Then we need a proper configuration file for our tests.

1. Install python modules

   $ poetry install
   

2. Create a configuration file
   Now, we copy the pre-defined development config file and amend a few things.
   This will copy the config and set the proper directory variables:

   $ sed -e "s;YOUR_AUR_ROOT;$PWD;g" conf/config.dev > conf/config
   

   For our tests we need a mariadb instance.
   We can actually make use of our mariadb container for this.
   Edit the config file conf/config and modify the database config part:

   ; MySQL database information. User defaults to root for containerized
   ; testing with mysqldb. This should be set to a non-root user.
   user = root
   password = aur
   host = 127.0.0.1
   port = 13306
   ;socket = /var/run/mysqld/mysqld.sock
   

3. Set environment variables
   We need to set a few environment variables for our test suites,
   the aur config file and the log config file:

   $ export AUR_CONFIG=conf/config
   $ export LOG_CONFIG=logging.test.conf
   

4. Compile translation & doc files

   $ make -C po install
   $ make -C doc
   

5. Running tests
   First make sure you have the mariadb container running.

   # docker compose start mariadb
   

   Now we can run our python test-suite or individual tests with:

   $ poetry run pytest test/
   $ poetry run pytest test/test_whatever.py
   

   To run Sharness tests:

   $ poetry run make -C test sh