Generic Adaptable Page Objects

page-objects-dont-have-to-suck.md

Goals:

• page objects should be cross-ecosystem, cross-technology
• simple API
• interop with Ember, and WebDriver (via adapters, which means testing-library would be supported, if anyone wanted to use that).
• all async/awaitable
• componentizable -- encourage separation of structure from the actual page object creation.
• all properties lazily evaluated

Nomenclature:

• Adapter: Converts meaning in to behavior -- "click" means different things to ember and webdriver
• Page Object: a literal object that represents how one would interact with the page

Proposed API:

import { setAdapter, create } from 'unknown package';
import { EmberAdapter } from 'unknown package/ember';

setAdapter(EmberAdapter);

const definition = {
  login: {
    username: '[data-test-username]',
    password: '[data-test-password]',
    submit: '[data-test-submit]',
  },

  list: {
    scope: '[data-test-list]',

    rows: collection('[data-test-row]', {
      click: '[data-test-row-clicker]'
    });
  }
};

const page = create(definition);

// Usage
await page.login.username.click();
page.login.username.isFocussed; // true
await page.login.username.fillIn('text'); // username is now "text"
page.login.username.dom // <input type='text' data-test-username ... >

page.login.submit.fillIn('text'); // Error!: buttons are not fillable
await page.login.submit.click();

// for typescript, types are optional
// by default, all methods and properties will be available, and will throw runtime errors
// but the type of the interactable will still auto-complete for all methods and properties.
interface Definition {
  login: {
    submit: ButtonInteractor;
  }
};
const typedPage = create<Definition>(definition);
// by default, every entry in the page object would be of type "OmniInteractor"

Usage with qunit-dom

assert.dom(page.login.username.dom).hasClass(...);

A few thoughts:

Adapters

Definitely a good idea. Questions:

scope

How are they scoped? I would propose globally, but replaceable. So the happy path would be calling setAdapter() somewhere global like test-helper.js in Ember. But if mixing different kinds of tests (similar to old Ember testing where acceptance and integration tests has different APIs for interacting with the DOM), could call setAdapter() in beforeEach? Another idea that's maybe overkill in the initial implementation but maybe worth considering is a pushAdapter()/popAdapter() API, which would allow calling from beforeEach/afterEach hooks, or using Ember's style, setupSomeKindOfPageObject(hooks);.

auto-selection

It seems like it wouldn't be too hard to detect certain test environments so you wouldn't have to explicitly set a test adapter if using a vanilla version of one of those environments. In this case, pushAdapter()/popAdapter() functionality is probably more important for overriding the auto-selection for specific tests.

Lazy vs. eager evaluation

Meaning, when accessing a page sub-object, do we immediately look up its DOM element, or wait until a method is invoked on it that requires its DOM element?

do we lazy evaluate?

ember-cli-page-object does, and It enables scenarios like:

let login = page.login; // DOM element doesn't exist yet, but `login` still evaluates to a page sub-object
await page.showLoginButton.click(); // now login DOM element exists
assert.equal(login.username.placeholder, 'username');
await login.username.fillIn('me');

I think we probably want lazy evaluation, especially because of the following scenario:

await page.showLoginButton.click();
let login = page.login;
login.username.fillIn('me');
await login.submit(); // shows loading UI, removing form from the DOM and then re-inserting it when loading is complete
// oops, since login form was removed and re-rendered, it's a different DOM element, so if we eager evaluated, we'd
// have to re-evaluate or something, or else login's DOM element would not be in the DOM and login !== page.login
assert.ok(login.somethingValidationErrorSomething);

collections

The lazy evaluation question gets complicated with collections. In your example above, what type does page.list.rows return? If it's an Array then we're not lazy evaluating. If we do lazy evaluate, then it has to be some array wrapper or proxy thingy or something. This is why ember-cli-page-object collections implement a few specific EmberArray-like methods (or maybe the full immutable interface, I forget). I've always found this a bit un-ergonomic, but don't have a great idea for how to improve it (since I'm very skeptical that eager evaluation would provide a better overall experience).

Writing page objects

properties/methods

It's not clear to me from your write-up what properties/methods are available on page sub-objects. In ember-cli-page-object, there are some default properties and methods available on all sub-objects (which I think they call components), but outside of those the page object author defines what's available on a sub-object by sub-object basis, including the name, e.g.

form {
  submit: clickable('input[type="submit"]')
}

where there is no page sub-object for the submit button, just an action on the form named submit that ends up clicking on a particular descendant DOM element of the form page (sub-)object's DOM element.

Part of the value proposition of ember-cli-page-object is the level of indirection allowed by this naming, sort of like behavior-driven testing, so if the functionality of your page doesn't change, but the UI gets significantly refactored you're more likely to be able to just reimplement your page object behind the same page object "API" and not have to change any of your tests. In practice, I've found this benefit of fairly limited value, as it tends to be more trouble than it's worth (or than it feels like it's worth) to write page objects whose behaviors are sufficiently abstracted from the DOM structure.

But anyway, this whole question is unclear to me from your snippet above. When I do

const definition = {
  bar: 'some-selector'
};
const page = create(definition);

who/what determines what properties/methods are available on page.bar?

custom properties/methods

Do we support custom properties? This is related to the above question, but if I want to implement the equivalent of a Javascript getter, where maybe I'm aggregating something across a collection of sub-elements, or accessing some really specific thing like calling bbox() on an SVG element, can I do that in the page object framework? Or do I just write a helper outside of the page object to get the value?

Similarly for compound actions? It's fairly common for me to do something like

const definition = {
  dropdown: {
    trigger: '[data-test-trigger]',
    items: collection('[data-test-item]'),

    async clickItem(index) {
      await this.trigger.click();
      await this.items[index].click();
    }
  }
};

Would we support that? I think if the framework did something similar to ember-cli-page-object and allows methods and getters on the definition, and invoked them with this properly set to the instantiated page object, it would work. But this should be defined/specified.

re-usable sub-objects/components

It's a pretty common use case to have a sub-object (or component in ember-cli-page-object's nomenclature) defined separately from a page, e.g. if it represents a reusable component rendered in multiple places in the application. Perhaps we just follow ember-cli-page-object's lead, which I think is implied by your example in having the definition be a separate structure from the created page object. But it's certainly worth being clear on this.

differences from ember-cli-page-object

Other than the framework-independence, what goals/requirements differ from ember-cli-page-object? Would it make more sense to use ember-cli-page-object as a starting point and remove/modify to meet differing requirements rather than green-fielding?

Adapters:

how are they scoped?

I'm thinking globally -- because I don't think it'd make sense to switch dom execution environments mid-test-run?

somewhere global like test-helper.js in Ember.

that's exactly what I was thinking!

In general, I think this sort of thing has to be global scope so that implementation can be minimal and also not require too much integration knowledge with other testing environments. the flexibility would be amazing, but I wouldn't want to enable people to lag behind upgrades or whatever. I'm a huge fan of everything being code-moddable, so, I'd hope no one ever feels they'd be left behind anyway?.

it seems like it wouldn't be too hard to detect certain test environments so you wouldn't have to explicitly set a test adapter

for sure! I imagine this sort of code could live in addon-space? or were you thinking of something else?

Lazy vs Eager:

Lazy, always. Resolving an entire page object would cause perf issues, as well as just being incorrect during async behaviour. your scenario is spot on.

collections:

• getting this right would be tricky -- I def would want some Array proxy type behavior, and not like what ec-po does. Everyone already knows native arrays.

It's not clear to me from your write-up what properties/methods are available on page sub-objects.

yeah, this is a big difference for me -- I want everything to be available, and fully typed (so that intellisense is available).
this would also enable you to re-export the definition so you could use the selectors directly, if you needed to, for whatever reason. basically, this just decouples the definition of yoru app from a specific page-object implementation.

who/what determines what properties/methods are available on page.bar?

page.bar would return an "Interactable" which just gives you all the APIs for clicking, filling in, typing, selecting, etc. Typescript would help (the royal) you out immensely here.

Do we support custom properties?

maybe? I think it would be up to the adapter to support.

Similarly for compound actions? It's fairly common for me to do something like

the definition would be unable to support that, but there could be an extended api -- maybe something like:

const page = create(definition, {
  login: {
    async doEverything() {
      await this.username.fillIn('...');
      await this.submit.click();
    }
  }
});

await page.login.doEverything();

or something like that.
but sense this is all definitiony, I wonder if the vanilla object should be renamed "structure", so that it maybe more closely correlates to semantics of the DOM, rather than what is defining your page object?

re-usable sub-objects/components

this is most certainly a goal. I'll add that.

Other than the framework-independence, what goals/requirements differ from ember-cli-page-object?
ember-cli-page-object uses jQuery, and doesn't seem to have any interest in getting rid of it or cleaning up its internals any time soon -- just seems like a non-priority to them :-\

Would it make more sense to use ember-cli-page-object as a starting point and remove/modify to meet differing requirements rather than green-fielding?

idk, the maintainer isn't on discord very often, so it's really hard to guage what the future of the library is, and if they are open to accepting breaking changes if it means better functionality in the long run.

I'm thinking globally -- because I don't think it'd make sense to switch dom execution environments mid-test-run?

I'm not sure. Previously it did make sense in Ember (when there were different DOM APIs in acceptance vs. integration tests). I don't know enough about other frameworks where this would be useful to know if that still does/might apply anywhere. If not, then yeah, probably a single global adapter should be enough.

I imagine this sort of code could live in addon-space? or were you thinking of something else?

I was thinking pretty fuzzily, but I think that makes sense -- an optional wrapper library (addon in Ember's case) to integrate it with frameworks with specific testing environments.

I think it would be up to the adapter to support [custom properties]

Is there an easy/convenient way for users to extend adapters then? In my experience, custom properties are an extremely common use case, and if the page sub-object expose any properties they will likely need to support custom user-defined (not framework-adapter-defined) properties.

And why can't the definition support custom actions? If the custom actions are moved to the page object (rather than the definition), then we'd need re-usable sub-objects to be page objects themselves rather than definitions, because they'd need to carry their custom actions with them.

re: ember-cli-page-object, back a couple of years ago when I was contributing heavily to the project, they were talking about a major release that cleans a whole bunch of stuff up and simplifies things -- see this tracking issue. I've found the maintainers fairly responsive on GitHub, and I imagine contributions would be welcome (and breaking changes in the context of a v2 as well). I'd float the idea over there of scooping the core out into a standalone/framework-independent library, since I think it makes a lot of sense.

However, I think you might need to get clearer on the goals here, or maybe you are but I'm just not getting them. I see a spectrum here with two extremes.

One extreme is just representing the DOM structure so selectors can be maintained in one place and nested in a structure so parent selectors don't have to be repeated. Then the page object would expose its tree structure, and outside of that it would only expose the dom property, which could be passed to qunit-dom and directly to @ember/test-helpers DOM helpers. So it's just a convenient API for defining the structure of a page and being able to access DOM elements in a rendered page.

The other extreme is full behavior-driven testing support where all the actions and properties and everything are abstracted. The happy path would be never using qunit-dom (but just "low-level" qunit assertions) or @ember/test-helpers or any DOM APIs at all, but of course it would need the "escape-hatch" of the dom property since this abstraction is never going to perfectly meet everyone's needs.

It's unclear to me where on that spectrum you're aiming to fall. I think ember-cli-page-object was aiming close to the latter, while you're aiming closer to the former, but not really all the way to the former...so I'm still unclear exactly what the vision is here. If I had to take a stab at what I think the best course of action would it, it would be to:

1. Simplify ember-cli-page-object a whole bunch (but still keep all of the primary functionality of components, custom properties, custom actions, etc.)
2. Clean up ember-cli-page-object's collections to have a more familiar array API
3. Enable use of qunit-dom (probably by just exposing a dom property on all elements, which is a pretty minor code change in ember-cli-page-object)
4. Separate the Ember-specific bits so the core (and vast majority of the code) lives in a framework-independent library

The point I was trying to convey (and thank you @bendemboski for articulating it much better then I) is that given:

[An object that] is just representing the DOM structure so selectors can be maintained in one place and nested in a structure so parent selectors don't have to be repeated [and the object] would expose its tree structure.

Then we are no longer talking about Page Objects I would suggest that this concept be moved away from the association of a Page Object as these two ideas are different things.

If, however we still want the Page Object concept then I would say a merger of ideas is best. Keep the Page Object abstraction and behaviour driven design but allow QUnit's assert to tap into the Page Object via qunit-dom for the better messaging/debugging.

Spitballing here: PageObject.assert(page.form.isVisible), page.form.assert('canLogin'), assert.dom(page).canViewLoginForm(), or something like that.

Separate the Ember-specific bits so the core (and vast majority of the code) lives in a framework-independent library

Using the current page-object API in non-ember projects would be heaven. Currently I have to write my own by hand.

class Page {
  constructor(testContext) {
    this.context = testContext;
  }
  find(selector) {
    return this.context.querySelector(`#my-scope ${selector}`);
  }
  get isVisible() {
    return this.find().style.display !== 'none' && ...;
  }
  doSomething() {
    this.find('button').dispatchEvent(new MouseEvent('click'));
  }
}

Total PITA.

Another great example of a good use of a framework agnostic API is Sinon. It does this by offering its own suite of assertions and then all you do is link the pass()/fail() to the test framework in question. In this way sinon can express complex assertions otherwise unavailable if it is done by hand. Another example would be chai.

I think @NullVoxPopuli's point is that QUnit's assertion API... ehhemm.... SUCKS compared to things like qunit-dom. It would be wonderful if e-c-p-o exposed a rich assertion API that could tap into QUnit/qunit-dom but keep the behavior expressiveness that Page Objects aspire to be.

I'm a big fan of most of the listed suggestions.

page objects should be cross-ecosystem, cross-technology

Yes. Adapters seems to be the most reasonable strategy to achieve this. I've made an attempt to extract Adapters in ember-cli-page-object. I like simpliefied API and possibilities it unlocks. However, in ec-page-object, it's currently possible to perform action against the multiple DOM elements, which makes it pretty hard to integrate adapters in V1 of ec-page-object.

pushAdapter()/popAdapter()

I'm afraid that can lead to an extra layer of indirection and learnability issues. Cause adapters detection can be order dependant.
I think setAdapter is good and clear enough. So in case, if you have a test suite mixed from, let's say, setupTest and moduleFor, you can set your mode explicitly before each test. And if you have a "clean" test suite, you can just do it once in the test-helper.js.

String as a scope

const definition = {
  login: {
    username: '[data-test-username]',
    password: '[data-test-password]',
    submit: '[data-test-submit]',
  },

I've explored this area a bit. I think it's a good idea.
But, in ec-page-object, we also have testContainer, which is also a string. Also we might want to add some string configuration in the future, which would not be a forward compatible feature..
I believe, in order to make it safe, we should change API to isolate all the query options like:

const definition = {
  withAStringScope: '.scope',

  withAScope: {
    scope: '.scope'
  },

  compWithAComplexQuery: {
    scope {
      selector '.scope',
      testContainer: '.testContainer'
    } 
  }
}

I think, with an ability to pass all the query options to the scope object we can safely treat all the string options as a scope.

@sukima your sinon framework agnostic idea looks interesting. I think it's worth to be explored.