tests.md

Frontend UI testing

The frontend UI testing tests whether the contents of Give's features display and function as intended. These tests don't focus on the aesthetics but rather tests the correct output on the webpage and the interactions with it.

How to run the tests?

A sample wordpress.sql is provided within sample-data/ folder which has few sample forms, donations and donors to test in various combinations.

Manual Testing

After setting up the local development environment, running tests manually is fairly simple. All you need it to run

npm run test

By default this will run the tests in the headless mode. If you wish to run the tests in non-headless mode, you can set

headless: false

inside tests/e2e/jest-puppeteer.config.js. This will launch an instance of Chrome browser and the tests will begin. You can also change the speed of execution by setting

slowMo: 50

The ideal speed range is between 30-80. Low number indicates high speed.

Automation testing

The automation testing architecture is set within Travis where all the frontend tests run inside a Docker Container for the current branch on every pull request. For every pull request, a Docker Container is created which installs WordPress and sets up the database using the aforementioned SQL file. After running the tests, the Docker Container is destroyed automatically.

How does it work?

Frontend UI tests in Give are written using Facebook's Jest Javascript Framework. These tests run on Puppeteer which is a headless Chrome Node API.

The tests in Jest run parallely by default, but in Give, each test has been configured to run sequentially to avoid opening multiple tabs in Chrome at the same time.

The Puppeteer API is not designed for testing. To address this and to make testing easier, Give uses an Open Source testing framework built over Jest and Puppeteer called Jest-Puppeteer, which exposes the assertion library for puppeteer. Most of the test cases uses functions provided by expect-puppeteer for ease of testing, and at some places it directly uses functions provided by Puppeteer API itself.

What does it test?

The frontend tests are bifurcated into 2 types

EXISTENCE TESTS

These tests are assertion tests which compare the expected output with the output found on the HTML DOM for a specific element. This also tests whether an HTML element that is expected to be part of the DOM is present in the DOM.

An example of existence test:

give.utility.fn.verifyExistence( page, [
	{
		desc: 'verify form title',
		selector: '.give-form-title',
		strict: true
		innerText: 'Simple Donation Form',
	}
])

The following 3 object properties describe the test, what to test, and how to test. These properties will be deleted from the object just before assertion test begins.

• desc: Desciption of the test. This will be output on the terminal screen.
• selector: The selector which needs to be tested.
• strict: Setting this to true will use toBe(), else it will use toMatch(). Default: false.

The 4th object property is innerText which is one of the many HTML node attributes. You can pass as many attributes you wish to test, it could be href, value and innerHTML; etc.

INTERACTION TESTS

These tests test the interaction with the webpage. This can be better visualized after setting the screenshot parameter to true which will generate a screenshot after every interaction.

It provides 3 types of interaction

• hover
• focus
• click

An example of interaction test:

give.utility.fn.verifyInteraction( page, [
	{
		desc: 'verify hover on title tooltip',
		selector: 'label[for="give-title"] .give-tooltip',
		event: 'hover',
	}
])

The above test will hover the mouse pointer over the label which has the for attribute set as give-title

How to write tests?

• Test files should end with .test.js suffix
• babel-jest is added to support ES6 syntax
• test-utility.js file contains helper functions and variables that should be used across all tests.
• The URL to test should be set within the beforeAll() method
• Priority should be given to expect-puppeteer followed by Puppeteer API. There are few bugs that produces race condition which causes tests to fail due to unresolved Promises. link#1, link#2, link#3, link#4

If the tests contain any action or event that might lead to redirection/navigation, for example after a form submission like:

page.click( '.form-submit' )
page.waitForNavigation()

This is known to cause a race condition. The following must be used as a workaround:

await Promise.all([
	page.click( '#give_login_submit' ),
	page.waitForNavigation()
])

Resources

1. jest-puppeteer
2. expect-puppeteer
3. Puppeteer API
4. Jest