Migrating from JSM to ES6 modules

migration.md

This document shows a possible migration path from Cu.import() to ES6 modules for the code of Firefox.

Add-ons and Thunderbird should not be affected by the migration.

1. Introduce ES6 modules as an alternative to Cu.import()

1. Make sure that calling ES6 import foo in chrome code always returns the same object regardless of the global from which it is called, as is the case currently with Cu.import().
2. Make sure that evaluating import foo in chrome code blocks the embedding as is the case currently with Cu.import().
3. Inform developers that they should now use ES6 modules instead of jsm for their new code.
4. Reject patches that introduce new jsm modules. Optionally, patch mozReview to do this automatically.

@jonco, how hard are 1. and 2.?

2. Introduce export in jsm

1. Patch mozJSComponentLoader to accept export { a, b, c } instead of EXPORTED_SYMBOLS when available. For compatibility with add-ons and Thunderbird, EXPORTED_SYMBOLS will remain usable as long as Cu.import() exists.
2. Start migrating the code from EXPORTED_SYMBOLS to export. An automatic rewrite should cover most cases.

@jonco, how ard is 1.?

3. Migrate Cu.import(foo) to import foo

1. Patch mozJSComponentLoader so that Cu.import(foo) and import foo from chrome code access the same singleton. Essentially, Cu.import() becomes a variant of import foo that returns a backstage pass instead of only the symbols actually exported. Keeping this compatibility will be necessary to ensure that we can migrate piecewise without breaking the unicity of module instances, as well as for compatibility with add-ons, Thunderbird.
2. Inform developers that they should now use import foo in their new code, rather than Cu.import().
3. Reject patches that introduce new instances of Cu.import(). Optionally, patch mozReview to do this automatically.
4. Start migrating the code from Cu.import() to import foo. An automatic rewrite should cover most cases.

@jonco How hard is 1?

4. Remaining cases

At this stage, we will still have the following cases to contend with:

• uses of XPCOM.defineLazyModuleGetter;
• conditional imports;
• scoped imports and imports from within a function.

As far as I can tell, none of these cases has a simple counterpart in ES6 modules. I believe, however, that once we have reached this stage, we will have improved considerably the state of our codebase and gained experience that we may then use to handle the missing cases.

Eventually, we may also want to get rid of backstage passes.

Limiting feedback to stage 1:

1. This would be nonstandard and not how modules are going to behave on the web. I'm not sure this should be the default behavior in chrome code.
2. The presence of import foo in Chrome code causes the whole module to be blocked until foo loads. This is more block-y than Cu.import().

Also note that ES import is not a function and you can't pass variables or arbitrary expressions to it, like

const Promise = Cu.import(PROMISE_URI, {}).Promise;
const { Loader } = Cu.import(PATH + 'toolkit/loader.js', {});

Most imports are pretty basic, and don't care about any of these details. For the special cases that do, the easiest way would be to keep Cu.import and have it support loading ES6 modules as well as jsms.

In response to @jorendorff:

1. This would be nonstandard and not how modules are going to behave on the web. I'm not sure this should be the default behavior in chrome code.

It is my understanding that this is the intended semantics of modules (although, yes, the implementation will need to differ): one instance of one application (== one page in one tab) only has one instance of each module, right?

I believe that the oddity here is the fact that we have XUL windows and XPCOM components and all sorts of containers (and globals) for a single application. I would say that they are the one that have non-web semantics but that they shouldn't bring down the whole ship with them.

1. The presence of import foo in Chrome code causes the whole module to be blocked until foo loads. This is more block-y than Cu.import().

True. I don't think that this is a problem in practice, though, because jsm code typically imports modules before starting. I'm planning to use Cu.import() as a crutch for lazy loading until we have a better solution.

Most imports are pretty basic, and don't care about any of these details. For the special cases that do, the easiest way would be to add a new Cu.importModule method that behaves like Cu.import, with the sole difference that it loads an ES module, not a JSM.

See above, I'm actually hoping that we can use Cu.import as a crutch and make it accept both ES modules and JSMs. Pending @Jonco's feedback.

OK, @Jonco came up with an approach that should work: only allow ES modules in one chrome global; make Cu.import() able to import ES modules and have it always load those modules in that global.

The rest of the plan seems reasonable except that in stage 2 I think ES modules should have a different file extension from JSMs, rather than try to implement ES export syntax in JSMs or otherwise make JSMs compatible with ES module syntax.

OK, @Jonco came up with an approach that should work: only allow ES modules in one chrome global; make Cu.import() able to import ES modules and have it always load those modules in that global.

Sounds good.

The rest of the plan seems reasonable except that in stage 2 I think ES modules should have a different file extension from JSMs, rather than try to implement ES export syntax in JSMs or otherwise make JSMs compatible with ES module syntax.

Mmmh... I would really like to migrate our existing JSM code to ES without breaking add-ons or Thunderbird, which rely upon a specific url for the jsm. This means that even after we have migrated a JSM code from url foo to an ES module, Cu.import(foo) needs to carry on working.

I see two ways to do it:

1. Progressively turn JSM modules into ES modules in-place, as outlined above.
2. Provide JSM shims for ES modules, possibly auto-generated as part of the build system.

The latter will not work for the Backstage Pass (or will require some annotations, changes in tests and a few pieces of build system hackery), but I guess I can live with that.

@jorendorff To be sure, what's the problem with implementing the export syntax in JSMs?