GeckoView has a lot of custom code that is used to run junit test. This document is an overview of what this code does and how it works.
GeckoView is an Android Library that can be used to embed Gecko, the Web Engine behind Firefox, in applications. It is the foundation for Firefox on Android and it is intended to be used to build Web Browsers, but can also be used to build other types of apps that need to display Web content.
GeckoView itself has no UI elements besides the Web View and uses Java interfaces called "delegates" to let embedders (i.e. apps that use GeckoView) implement UI behavior.
For example, when a Web page's JavaScript code calls alert('Hello') the
embedder will receive a call to the onAlertPrompt method of the
PromptDelegate interface with all the information needed to display the
prompt.
As most delegate methods deal with UI elements, GeckoView will execute them on the UI thread for the embedder's convenience.
One thing that is important to understand for what follows is GeckoResult.
GeckoResult is a promise-like object that is used throughout the GeckoView
API, it allows embedders to asynchronously respond to delegate calls and
GeckoView to return results asynchronously. This is especially important for
GeckoView as it never provides synchronous access to Gecko as a design
principle.
For example, when installing a WebExtension in GeckoView, the resulting
WebExtension object is returned in a GeckoResult which is completed when
the extension is fully installed:
public GeckoResult<WebExtension> install(...)
To simplify memory safety, GeckoResult will always execute callbacks in the
same thread where it was created, turning asynchronous code into
single-threaded javascript-style code. This is currently implemented using the
Android Looper for the thread, which restricts GeckoResult to threads that
have a looper, like the Android UI thread.
Given that GeckoView is effectively a translation layer between Gecko and the embedder, it's mostly tested through integration tests. The vast majority of the GeckoView tests are of the form
- Load simple test web page
- Interact with the web page through a privileged JavaScript test API
- Verify that the right delegates are called with the right inputs
and most of the test framework is built around making sure that these interactions are easy to write and verify.
Tests in GeckoView can be run using the mach interface, which is used by most
Gecko tests. E.g. to run the loadUnknownHost test in NavigationDelegateTest
you would type on your terminal:
./mach geckoview-junit org.mozilla.geckoview.test.NavigationDelegateTest#loadUnknownHost
Another way to run GeckoView tests is through the Android Studio IDE. By running tests this way, however, some parts of the test framework are not initialized, and thus some tests behave differently or fail, as will be explained later.
Being a library, GeckoView has a natural, stable, testing envelope, namely the GeckoView API. The vast majority of GeckoView tests only use publicly-accessible APIs to verify the behavior of the API.
Whenever the API is not enough to properly test behavior, the testing framework offers targeted "privileged" testing APIs.
Using a restricted, stable testing envelope has proven over the years to be an effective way of writing consistent tests that don't break upon refactoring.
When run through mach, the GeckoView junit tests run in a similar environment
as mochitests (a type of Web regression tests used in Gecko). They have access
to the mochitest web server at example.com, and inherit most of the testing
prefs and profile.
Note the environment will not be the same as mochitests when the test is run through Android Studio, the prefs will be inherited from the default GeckoView prefs (i.e. the same prefs that would be enabled in a consumer's build of GeckoView) and the mochitest web server will not be available.
Tests account for this using the confusingly named isAutomation check,
which essentially checks whether the test is running under mach or via
Android Studio.
Unlike most other junit tests in the wild, GeckoView tests run in the UI thread. This is done so that the GeckoResult objects are created on the right thread. Without this, every test would most likely include a lot of blocks that run code in the UI thread, adding significant boilerplate.
Running tests on the UI thread is achieved by registering a custom TestRule
called GeckoSessionTestRule, which, among other things, overrides the
evaluate method and wraps everything into a
instrumentation.runOnMainSync call.
As mentioned earlier, verifying that a delegate call happens is one of the most
common assertions that a GeckoView test makes. To facilitate that,
GeckoSessionTestRule offers several delegate* utilities like:
sessionRule.delegateUntilTestEnd(...)
sessionRule.delegateDuringNextWait(...)
sessionRule.waitUntilCalled(...)
sessionRule.forCallbacksDuringWait(...)
These all take an arbitrary delegate object (which may include multiple delegate implementations) and handle installing and cleaning up the delegate as needed.
Another set of facilities that GeckoSessionTestRule offers allow tests to
synchronously wait* for events, e.g.
sessionRule.waitForJS(...)
sessionRule.waitForResult(...)
sessionRule.waitForPageStop(...)
These facilities work together with the delegate* facilities by marking the
NextWait or the DuringWait events.
As an example, a test could load a page using
session.loadUri, wait until the page has finished loading using
waitForPageStop and then verify that the expected delegate was called using
forCallbacksDuringWait.
Note that the DuringWait here always refers to the last time a wait* method
was called and finished executing.
The next sections will go into how this works and how it's implemented.
One thing you might have noticed in the above section is that
forCallbacksDuringWait moves "backward" in time by replaying the delegates
called that happened while the wait was being executed. GeckoSessionTestRule
achieves this by injecting a proxy object into every delegate, and
proxying every call to the current delegate according to the delegate
test calls.
The proxy delegate is built using the Java reflection's
Proxy.newProxyInstance method and receives a callback every time a
method on the delegate is being executed.
GeckoSessionTestRule maintains a list of "default" delegates used in
GeckoView, and will use reflection to match the object passed into the
delegate* calls to the proxy delegates.
For example, when calling
sessionRule.delegateUntilTestEnd(object : NavigationDelegate, ProgressDelegate {})
GeckoSessionTestRule will know to redirect all NavigationDelegate and
ProgressDelegate calls to the object passed in delegateUntilTestEnd.
Some delegate methods require output data to be passed in by the embedder, and this requires extra care when going "backward in time" by replaying the delegate's call.
For example, whenever a page loads, GeckoView will call
GeckoResult<AllowOrDeny> onLoadRequest(...) to know if the load can continue
or not. When replaying delegates, however, we don't know what the value of
onLoadRequest will be (or if the test is going to install a delegate for it,
either!).
What GeckoSessionTestRule does, instead, is to return the default value
for the delegate method, and ignore the replayed delegate method return value. This can
be a little confusing for test writers, for example this code will not stop
the page from loading:
session.loadUri("https://www.mozilla.org")
sessionRule.waitForPageStop()
sessionRule.forCallbacksDuringWait(object : NavigationDelegate {
override fun onLoadRequest(session: GeckoSession, request: LoadRequest) :
GeckoResult<AllowOrDeny>? {
// this value is ignored
return GeckoResult.deny()
}
})
as the page has already loaded by the time the forCallbacksDuringWait call is
executed.
To track when a wait occurs and to know when to replay delegate calls,
GeckoSessionTestRule stores the list of delegate calls in a
List<CallRecord> object, where CallRecord is a class that has enough
information to replay a delegate call. The test rule will track the start and
end index of the last wait's delegate calls and replay it when
forCallbacksDuringWait is called.
To wait until a delegate call happens, the test rule will first examine the
already executed delegate calls using the call record list described above. If
none of the calls match, then it will wait for new calls to happen, using
UiThreadUtils.waitForCondition.
waitForCondition is also used to implement other type of wait* methods like
waitForResult, which waits until a GeckoResult is executed.
waitForCondition runs on the UI thread, and it synchronously waits for an
event to occur. The events it waits for normally execute on the UI thread as
well, so it injects itself in the Android event loop, checking for the
condition after every event has executed. If no more events remain in the
queue, it posts a delayed 100ms task to avoid clogging the event loop.
As you might have noticed from an earlier section, the test rule allows tests
to run arbitrary JavaScript code using waitForJS. The GeckoView API, however,
doesn't offer such an API.
The way waitForJS and evaluateJS are implemented will be the focus of this
section.
The only supported way of accessing a web page for embedders is to write a built-in WebExtension and install it. This was done intentionally to avoid having to rewrite a lot of the Web-Content-related APIs that the WebExtension API offers.
GeckoView extends the WebExtension API to allow embedders to communicate to the extension by overloading the native messaging API (which is not normally implemented on mobile). Embedders can register themselves as a native app and the built-in extension will be able to exchange messages and open ports with the embedder.
This is still a controversial topic among smaller embedders, especially solo developers, and we have discussed internally the possibility to expose a simpler API to run one-off javascript snippets, similar to what Chromium's WebView offers, but nothing has been developed so far.
To run arbitrary javascript in GeckoView, the test runner installs a support extension.
The test framework then establishes a port for the background script, used to run code in the main process, and a port for every window, to be able to run javascript on test web pages.
When evaluteJS is called, the test framework will send a message to the
extension which then calls eval on it and returns the
JSON-stringified version of the result back to the test framework.
The test framework also supports promises with evaluatePromiseJS. It works
similarly to evaluateJS but instead of returning the stringified value, it
sets the return value of the eval call into the this object, keyed by a
randomly-generated UUID.
this[uuid] = eval(...)
evaluatePromiseJS then returns an ExtensionPromise Java object which has a
getValue method on it, which will essentially execute await this[uuid] to get the value from the promise when needed.
A natural way of breaking the boundaries of the GeckoView API, is to run a so-called "experiment extension". Experiment extensions have access to the full Gecko front-end, which is written in JavaScript, and don't have limits on what they can do. Experiment extensions are essentially what old add-ons used to be in Firefox, very powerful and very dangerous.
The test runner uses experiments to offer privileged APIs to tests like
setPref or getLinkColor (which is not normally available to websites for
privacy concerns).
Each privileged API is exposed as an ordinary Java API and the test framework doesn't offer a way to run arbitrary chrome code, to discourage developers from relying too much on implementation-dependent privileged code.