Best_Practices.md

Best Practices for Writing tests

A test isn't given much though until it fails. Anybody should be able to look at a test and understand what and why it failed. Tests should be written for the failure case. Cypress creates an outline in the GUI to help us understand setup and what a test does. Well written tests help us understand what failed. Screenshots and video help us understand how it failed.

Before continuing, read Best Practices from the Cypress documentation first.

Each test must be runnable in complete isolation

A way to decrease test suite run time is to increase the amount of tests being run at the same time. Sharding (or splitting or work) typically happens at the file level, so at minimum a test file should not rely on any state from a previous test file.

Tests can also fail at any step. Each it block is run regardless of previously run it blocks. If one it block relies on the state left by another it block, cascading failures can happen that are very difficult to understand. Use before to run once per describe block and beforeEach to run multiple times

// bad
describe('Searches Page', () => {
  it('should load the searches page', () => {
    before(() => {
      // visit searches page
    })
  })

  it('should click the "My Favorites Tab"', () => {
    // Click "My Favorites Tab"
  })

  it('should have the correct column names on the My Favorites Tab', () => {
    // Assert column names
    // whoops, if the previous test fails, this will also fail
  })

  it('should click the "All Searches Tab"', () => {
    // Click "All Searches Tab"
  })

  it('should have the correct column names on the All Searches Tab', () => {
    // Assert column names
    // whoops, if the previous test fails, this will also fail
  })
})

// good
describe('Searches Page', () => {
  // run once for the entire "Searches Page" block
  // we use `before` instead of `beforeEach` for speed because our tests can handle it only be run once
  before(() => {
    // visit searches page
  })

  describe('My Favorites Tab', () => {
    // Run for every `it` block in the "My Favorites Tab" block
    beforeEach(() => {
      // Make sure we are in the right state for each `it` block
      // Click the "All Searches Tab"
    })

    it('should have the correct column names', () => {
      // Assert column names
    })
  })
})

A test should have limited focus

If you have to use the word "and" or a comma, chances are the test is doing too much. Failures will be tougher to spot. Developers may later add more to a test that lacks focus further making failures harder to understand.

Cypress will create an outline in the Command log showing describe, context and it blocks and how they nest. It helps to have short and concise messages to help understand this outline.

Breaking out tests may take longer to run, but understanding failures is more important. More hardware can be thrown at your test suite to decrease run time. A human spending more time to figure out failures is not only more expensive, but it also means he or she is spending less time doing tasks only humans can do.

// bad - duplicate information and hard to separate setup
describe('Verify expected columns under different searches tabs', () => {
  it('should verify expected columns under My Saved, All Searches, Shared Searches and My Favorite tabs', () => {
    // setup and assertions
    // maybe somebody later puts more here because there doesn't seem to be any focus
  })
})
// Outline:
// Verify expected columns under different searches tabs
//   - should verify expected columns under My Saved, All Searches, Shared Searches and My Favorite tabs

// good
describe('Searches Page', () => {
  before(() => {
    // visit searches page
  })

  describe('My Saved Tab', () => {
    beforeEach(() => {
      // click "My Saved" tab
    })

    it('should have the correct column names', () => {
      // verify column names for "My Saved Tab"
    })

    // This seems like a nice place to add another test that need the "My Saved Tab" active
  })

  describe('All Searches Tab', () => {
    beforeEach(() => {
      // click "All Searches" tab
    })

    it('should have the correct column names', () => {
      // verify column names for "All Searches Tab"
    })
  })
})
// Outline:
// Searches Page
//  - My Saved Tab
//    - should have the correct column names
//  - All Searches Tab
//    - should have the correct column names

Assertions should be obvious

Cypress uses Chai and also includes two extra matcher suites: chai-sinonhttp://www.chaijs.com/plugins/chai-sinon/ and chai-jquery. The Cypress documentation lists them all here: https://docs.cypress.io/guides/references/assertions.html. A common problem is expecting something to be true which leads to unhelpful assertion failures. Bad assertions prevents using cy.should shorthand. Finding the right matcher does take more time, but the failure messages help narrow down what failed.

A good practice is to force an assertion to fail and see if the error message and the Cypress Log output is enough to know why. It is easiest to put a .only on the it block you're evaluating. This way the application will stop where a screenshot is normally taken and you're left to debug as if you were debugging a real failure. This practice will help you write better tests.

// bad
cy.get('body').should($body => {
  expect($body.find('[data-testid=Tab]').length === 2).to.equal(true) // expected false to equal true
})

// better
cy.get('body').should($body => {
  expect($body).to.have.descendants('[data-testid=Tab]') // expected '<body>' to have descendants '[data-testid=Tab]'
  expect($body.find('[data-testid=Tab]')).to.have.length(2) // expected '[ <div[data-testid=Tab]>, 4 more... ]' to have a length of 2 but got 5
})

// best - we can use the shorthand
cy
  .get('body')
  .should('have.descendants', '[data-testid=Tab]') // expected '<body>' to have descendants '[data-testid=Tab]'
  .find('[data-testid=Tab]')
  .should('have.length', 2) // expected '[ <div[data-testid=Tab]>, 4 more... ]' to have a length of 2 but got 5