Brno bootcamp syllabus

tripleo-ansible-role-syllabus.md

Developing a new role

Developing a new role is simple in TripleO-Ansible. The TripleO-Ansible software developer kit, within the code repository, provides for everything needed to generate documentation, tests and role a wire-frame. TripleO-Ansible uses a playbook to create new roles, add the new role to the ZuulCI job configuration, and document the new role.

Running the role generation playbook

To run the role generation playbook, run ten simple commands.

1. Install package dependencies.

sudo yum install gcc python*-virtualenv

2. Ensure the python executable is available (optional)

if [[ ! -f /usr/bin/python ]]; then
  sudo ln -s /usr/bin/python3 /usr/bin/python
fi

3. Create a new python virtual environment for the molecule tests.

python3 -m virtualenv ~/test-python

4. With a clean development environment, clone the TripleO-Ansible repository.

git clone https://opendev.org/openstack/tripleo-ansible ~/tripleo-ansible

5. Change directory to the newly cloned repository.

cd ~/tripleo-ansible

6. Install all of the python requirements into the virtual environment.

~/test-python/bin/pip install ansible

7. Create a new branch.

git checkout -b tripleo-hello-world

8. Activate the venv.

source ~/test-python/bin/activate

9. Generate a new role.

~/test-python/bin/ansible-playbook -i localhost, role-addition.yml -e role_name=tripleo-hello-world

10. Deactivate the venv

deactivate

This playbook will generate the new role skeleton, create the documentation page for the new role, and add the new role to the ZuulCI configuration. The developer can then change directory to the newly generated role, ~/tripleo-ansible/tripleo_ansible/roles/tripleo-hello-world, and begin work.

Testing roles within TripleO-Ansible

Running a simple test within TripleO-Ansible requires nothing more than a vacant OS running CentOS or Fedora (RHEL8 comming soon). Simple test instances as provided by libvirt, vagrant, virtualbox, etc are more than enough to run tests with a clean system.

The TripleO-Ansible project is gated with ZuulCI and all code review is done within Gerrit. The code review portal and testing infrastructure is managed by OpenStack Infrastructure.

Running local, like ZuulCI

Testing TripleO-Ansible roles can be performed locally in the same way as ZuulCI. This functionality gives developers a simple way to develop new role functionality on local test instances, which allows developers the ability to create content with confidence that it will pass the gate.

Breaking down role tests

Every role within TripleO-Ansible is tested independently from every other role. While the code repository is a monolith, the testing framework is anything but. Tests are broken down in two catagories, molecule, and scenario.

• Molecule tests are triggered anytime a change is made to a given role and every role has its own specific set of molecule tests.

• Integration tests run through a full deployment, ensuring we're maintaining functionality and system integration, within the deployment framework.

Whenever a review is submitted to the TripleO-Ansible project ZuulCI matches the change to limited set of jobs. The job matching functionality ensures we're testing only what is necessary to provide developer feedback and give reviews high confidence the being reviewed is sound. It is expected that a single review submitted to a single role will trigger at least one molecule job and one integration job.

If a review spans more than one role, multiple molecule and integration jobs may be triggered.

Molecule testing overview

Molecule tests are built to exercise every code path within a role. Each role may have one or molecule scenarios which run various code path's within a role. At a minimum each role will have the default scenario. This scenario runs through the default role code path.

Integration testing overview

Integration tests are maintained by the TripleO-CI. TripleO-Ansible uses the integration jobs to ensure no review breaks expected functionality. These jobs will run an end to end deployment, using either multi-node or standalone test environments.

Running local tests

Running local tests is simple and can be done with as little as two commands.

1. Change directory to the newly cloned repository.

cd ~/tripleo-ansible

2. Run the local test script with the name of a given role being tested.

./scripts/run-local-test tripleo-hello-world

The script will run all molecule tests with a given role, in the same way ZuulCI would. This interface gives developers a simple means to test roles and role changes easily.

Running manual tests

Tests can be run manually using molecule from within the role directory. This functionality is considered "advanced" as it will require the role developer to know more about the test interactions and the molecule CLI utility. While "advanced" it is extreamly useful for debugging specific parts of a test run and can produce very interesting results.

Running molecule tests requires a little more setup when running from a clean workstation, however it can be done in as little as twelve steps.

1. With a clean development environment, clone the TripleO-Ansible repository.

This command is not required if there's already a checkout on the local workstation.

git clone https://opendev.org/openstack/tripleo-ansible ~/tripleo-ansible

2. Install package dependencies.

yum install gcc python*-virtualenv

3. Create a new python virtual environment for the molecule tests.

python3 -m virtualenv ~/test-python

4. Install all of the python requirements into the virtual environment.

~/test-python/bin/pip install -r ~/tripleo-ansible/requirements.txt -r ~/tripleo-ansible/molecule-requirements.txt

5. Change your user to yourself

You can also log out and log back in.

sudo su - $USER

This is only required because our user will need access to the Docker process.

6. Activate the test-python virtual environment.

source ~/test-python/bin/activate

7. Install any system packages using the bindep installation script.

~/tripleo-ansible/scripts/bindep-install

8. Source the test ansible environment variables.

source ~/tripleo-ansible/ansible-test-env.rc

9. Change directory to the role within the newly cloned repository.

cd ~/tripleo-ansible/tripleo_ansible/roles/tripleo-hello-world

10. Run the molecule test command.

molecule test --all

11. Get coffee

The molecule test command will execute all tests within a given role, giving the developer direct feedback about the role. While longer than running the local test script, this method gives developers direct access into the test functions of a role, which can be very useful when debugging hard issues, or developing new tests.

12. Deactivate the venv

deactivate

Testing review

When testing with molecule the test should aim to be quick and simple. An example molecule test should finish in as little as 1 minute. Depending on what is being tested, a molecule run could take up to 45 minutes; this is the ZuulCI upper limit. In general, if the test scenario takes longer than 45 minutes the tests should be broken down into smaller molecule scenarios. If smaller scenarios are not possible, tests should be performed using the integration test suit, which has a time limit of up to 3 hours.

Checking for lint

Lint checking in TripleO-Ansible is all run through the OpenStack standard tox. To run lint checks on the code base simply invoke tox with the linters environment option. While tox can be installed in any number of ways, it is recommended to create a new virtual environment to ensure you have the latest tox installed prior to running the tests.

The following six steps will do everything needed to lint check the code base.

1. With a clean development environment, clone the TripleO-Ansible repository.

This command is not required if there's already a checkout on the local workstation.

git clone https://opendev.org/openstack/tripleo-ansible ~/tripleo-ansible

2. Create a new virtual environment.

python3 -m virtualenv ~/tox

3. Install tox.

~/tox/bin/pip install tox

4. Activate the virtual environment.

source ~/tox/bin/activate

5. Change directory to the newly cloned repository.

cd ~/tripleo-ansible

6. Run the linters test.

tox -e linters

The linters check will validate the code-bases passes our coding standards. This includes yaml-lint, ansible-lint, flake8, bashate, and a check for any broken symlinks. These tests take a couple minutes but will ensure that there are not known syntax issues being introduced into the code-base.

Documentation

In the case of new role development, the documentation source code is contained under, doc/source/roles/. All role documentation files are prefixed with role-. As mentioned, the auto-documentation plugin will also parse modules. The source code for the module documentation is contained under doc/source/modules/, and uses a prefix of the plugin type, (action-, callback-, modules-, etc.).

Creating documentation

Creating documentation is simple and requires only a simple shim to get started.

1. Create the documentation reStructuredText file.

This command is not required if the file already exists.

cat > doc/source/roles/role-tripleo-hello-world.rst <<EOF
==========================
Role - tripleo-hello-world
==========================

.. ansibleautoplugin::
   :role: tripleo_ansible/roles/tripleo-hello-world
EOF

2. With the file in place, edit the file to add additional information to the role documentation as needed.

As you may notice, there is an autodoc plugin being used to create the role documentation. While extremely powerful, this plugin is not the only source of documentation developers should rely on. If additional information is required or useful to the end-user, additional documentation can be added to any role in reStructuredText format.

Generating the documentation

When submitting changes to TripleO-Ansible, ZuulCI will generate documentation anytime files within the docs/ path are changed. Documentation is generated using the OpenStack standard tox through sphinx. To ease in the creation of documentation the TripleO-Ansible project employees a sphinx plugin which will parse Ansible roles and modules, and generate documentation from it's derived information.

3. Run the docs generation.

tox -e docs

5. Change directory to the built html documentation.

cd ~/tripleo-ansible/doc/build/html

6. Get your server IP address

ip -o r g 1 | awk '{print $7}'

7. Start a simple html server so we can browse the documentation.

You may need to add a firewall rule sudo iptables -I INPUT 1 -p tcp --dport 12345 -j ACCEPT.

python -m http.server 12345 --bind $(ip -o r g 1 | awk '{print $7}')

Open a browser window and navigate to your servers IP address on port 12345.

8. Stop the simple http server with CTRL-C.

9. Deactivate the venv

deactivate

Create a new playbook for our role

The roles we create can be used as-is. All of the roles we're building can be used without the TripleO deployment framework. This means we can use our roles in playbooks to operate systems, deploy services, or otherwise run tasks; effectively enabling operators, developers, and deployers to make use of ansible, directly.

Building a playbook is simple.

1. Create a playbook file referencing the role.

cat > ~/tripleo-ansible/tripleo_ansible/playbooks/tripleo-hello-world.yaml <<EOF
---
- hosts: localhost
  connection: local
  roles:
    - role: tripleo-hello-world
EOF

2. Activate the test-python virtual environment.

source ~/test-python/bin/activate

3. Source the test ansible environment variables.

source ~/tripleo-ansible/ansible-test-env.rc

4. Run our new playbook

ansible-playbook -i 'localhost,' ~/tripleo-ansible/tripleo_ansible/playbooks/tripleo-hello-world.yaml

Given we can see our role works, lets make our playbook do something interesting. We'll set the timezone of our host to UTC. To do this we'll run the tripleo-timezone role in our new playbook.

5. Update our new playbook.

---
- hosts: localhost
  connection: local
  roles:
    - role: tripleo-hello-world
    - role: tripleo-timezone
      tripleo_timezone: UTC

6. Re-run our new playbook

ansible-playbook -i 'localhost,' ~/tripleo-ansible/tripleo_ansible/playbooks/tripleo-hello-world.yaml

Once again, lets update our playbook to make it do more with our existing roles. Lets setup the SSH config, enable a message of the day and create a banner.

7. Update our new playbook.

---
- hosts: localhost
  connection: local
  roles:
    - role: tripleo-hello-world
    - role: tripleo-timezone
      tripleo_timezone: UTC
    - role: tripleo-sshd
      tripleo_sshd_motd_enabled: true
      tripleo_sshd_banner_enabled: true

8. Once again re-run our new playbook.

ansible-playbook -i 'localhost,' ~/tripleo-ansible/tripleo_ansible/playbooks/tripleo-hello-world.yaml

---

Thats all folks!

Happy hacking.