This gist had a far larger impact than I imagined it would, and apparently people are still finding it, so a quick update:
- TC39 is currently moving forward with a slightly different version of TLA, referred to as 'variant B', in which a module with TLA doesn't block sibling execution. This vastly reduces the danger of parallelizable work happening in serial and thereby delaying startup, which was the concern that motivated me to write this gist
- In the wild, we're seeing
(async main(){...}())as a substitute for TLA. This completely eliminates the blocking problem (yay!) but it's less powerful, and harder to statically analyse (boo). In other words the lack of TLA is causing real problems - Therefore, a version of TLA that solves the original issue is a valuable addition to the language, and I'm in full support of the current proposal, which you can read here.
I'll leave the rest of this document unedited, for archaeological reasons.
Follow-ups:
- Why imperative imports are slower than declarative imports
- Dynamic module loading done right
- Non-deterministic module ordering is an even bigger footgun
As the creator of Rollup I often get cc'd on discussions about JavaScript modules and their semantics. Recently I've found myself in various conversations about top-level await.
At first, my reaction was that it's such a self-evidently bad idea that I must have just misunderstood something. But I'm no longer sure that's the case, so I'm sticking my oar in: Top-level await, as far as I can tell, is a mistake and it should not become part of the language. I'm writing this in the hope that I really have misunderstood, and that someone can patiently explain the nature of my misunderstanding.
ES2017 will introduce async and await, which make it much easier to write a series (take a mental note of that word, 'series') of asynchronous operations. To borrow from Jake:
// this Promise-based code...
function loadStory() {
return getJSON('story.json').then(function(story) {
addHtmlToPage(story.heading);
return story.chapterURLs.map(getJSON)
.reduce(function(chain, chapterPromise) {
return chain.then(function() {
return chapterPromise;
}).then(function(chapter) {
addHtmlToPage(chapter.html);
});
}, Promise.resolve());
}).then(function() {
addTextToPage("All done");
}).catch(function(err) {
addTextToPage("Argh, broken: " + err.message);
}).then(function() {
document.querySelector('.spinner').style.display = 'none';
});
}
// ...becomes this:
async function loadStory() {
try {
let story = await getJSON('story.json');
addHtmlToPage(story.heading);
for (let chapter of story.chapterURLs.map(getJSON)) {
addHtmlToPage((await chapter).html);
}
addTextToPage("All done");
} catch (err) {
addTextToPage("Argh, broken: " + err.message);
}
document.querySelector('.spinner').style.display = 'none';
}Lovely. The intent is much clearer, and the code is more readable. You can use async and await in modern browsers via async-to-gen, or, if you need to support older browsers and don't mind a bit more transpiled code, via Babel with the ES2017 preset.
Note that await can only be used inside an async function. Top-level await is a proposal to allow await at the top level of JavaScript modules.
Yes, it does, at first. One of the things some people don't like about JavaScript modules is that you can't load modules dynamically – whereas in Node.js you can do this...
var x = condition ? require( './foo' ) : require( './bar' );
doSomethingWith( x );...there's no JavaScript module equivalent, because import declarations are entirely static. We will be able to load modules dynamically when browsers eventually support them...
// NB: may not look exactly like this
import( condition ? './foo.js' : './bar.js' ).then( x => {
doSomethingWith( x );
});...but as you can see, it's asynchronous. It has to be, because it has to work across a network – it can't block execution like require does.
Top-level await would allow us to do this:
const x = await import( condition ? './foo.js' : './bar.js' );
doSomethingWith( x );Edit: it's actually worse than I thought – see this follow-up
The problem here is that any modules that depend on modules with a top-level await must also wait. The example above makes this seem harmless enough, but what if your app depended on this module?
// data.js
const data = await fetch( '/data.json' ).then( r => r.json() ).then( hydrateSomehow );
export default data;You've just put your entire app – anything that depends on data.js, or that depends on a module that depends on data.js – at the mercy of that network request. Even assuming all goes well, you've prevented yourself from doing any other work, like rendering views that don't depend on that data.
And remember before, when I asked you to make a note of the word 'series'? Since module evaluation order is deterministic, if you had multiple modules making similar network requests (or indeed anything asynchronous) those operations would not be able to happen in parallel.
To illustrate this, imagine the following contrived scenario:
// main.js
import './foo.js';
import './bar.js';
// foo.js
await delay( Math.random() * 1000 );
console.log( 'foo happened' );
// bar.js
await delay( Math.random() * 1000 );
console.log( 'bar happened' );What will the order of the logs be? If you answered 'could be either', I'm fairly sure you're wrong – the order of module evaluation is determined by the order of import declarations.
As far as I can tell (and I could be wrong!) there's simply no way for a CommonJS module to require a JavaScript module with a top-level await. The path towards interop is already narrow enough, and we really need to get this right.
No, you won't. You'll educate some of them, but not all. If you give people tools like with, eval, and top-level await, they will be misused, with bad consequences for users of the web.
Hogwash. There's nothing wrong with this:
import getFoo from './foo.js';
import getBar from './bar.js';
import getBaz from './baz.js';
async function renderStuff () {
const [ foo, bar, baz ] = await Promise.all([ getFoo(), getBar(), getBaz() });
doSomethingWith( foo, bar, baz );
}
renderStuff();
// code down here can happily execute while all three
// network requests are taking place
doSomeOtherStuffWhileWeWait();The version that uses top-level await – where network requests happen serially, and we can't do anything else while they happen – is slightly nicer, but certainly not to a degree that justifies the degraded functionality:
import foo from './foo.js';
import bar from './bar.js';
import baz from './baz.js';
doSomethingWith( foo, bar, baz );
// code down here has to wait for all the data to arrive
apologiseToTheUserForMakingThemWait();True, dynamic module loading is a bit trickier in this context (though still absolutely possible – also, see dynamic module loading done right). Robert Palmer suggests that a compromise would be to allow the await keyword at the top level but only for await import(...), not anything else. I'm ambivalent about having a keyword mean slightly different things in different places, but this seems worth exploring.
Say that to my face.
No, I'm not – I don't believe that tooling should drive these sorts of conversations (though at the same time, great things can happen when language designers think deeply about tooling and workflow). But yes, since you asked, top-level await probably will make it harder for tools like Rollup to create really tightly-optimised bundles of code.
Lots of smarter and more experienced developers than me seem to think top-level await is a great idea, so perhaps it is just me. It seems unlikely that none of this has occurred to people on TC39. But then again maybe joining those echelons involves a transformation that makes it hard to relate to mortals, like Jon Osterman becoming Doctor Manhattan.
Hopefully someone will be able to explain which it is.
I have to disagree with this sarcastic sentiment. It was trying to fool proof things and babysit over educating that broke async/await. If you can remove pitfalls without compromise then fair enough but otherwise you're just busting up the system in a lost cause to save the kind of person that wouldn't be safe in a padded room. The language can become a laughing stock as it starts to take on the form of its most hamfisted users sinking to their level rather than doing the best that can be done to level them up and being done with it. People will make mistakes, you shouldn't feel responsible all the time. There's a threshold somewhere that if you cross you'll be trying to maintain a standard that's impossible.
I'm not sure I see any definite problems presented, none for me anyway.
I would generally agree with the sentiment however that for front end where people are far more likely to use it such as with your example for xhr stuff and they're going to potentially cause some very annoying problems with error handling. I bet you someone will have
import 'when_ready';as a lazy way to run the script only when the body is ready. That will block all dependants from running and someone will have the wrong program flow already so people will do weird things like import to resume running something else not needing to wait for ready. Now that I have suggested it, it will happen.Such people mess things up anyway and I've long reached the point of just fixing it or throwing it out. Routinely working on millions of lines of code of legacy often from people who do not know how to program paints a very different and realistic picture of the pointlessness of haggling over something that is going to do nothing to cure all ills in software development, it wont even be a drop in the ocean. That's what DailyWTF is for though so it's all in hand.
The way static imports work, you're not really supposed to be depending on order. I'd like to see the use case for that.
Normally if b has to be after a then rather import a then b, a would import b. I think the only case I have like that is potentially of variable dependency and a global, such as which window object to use which has to be set to global because of scripts you have to use but that you can't plug it into.
I already developed my own system for async loading. If I put !async at the end of an import it assumes a module with only a default and awaits until it's no longer a promise. Ironically though, there seems to be a bug with this ATM when loading a specific CJS script (possibly while also loading the whole module elsewhere). I don't mind little CJS to MJS but broken MJS to CJS might be more problematic. I'd hope for MJS for anything new and old would not tend to depend on newer so CJS to MJS would not want to be as extensive.
Funnily enough my system also utilises rollup. It has a set of generic builders which rollup uses and a custom loader. However, the !async (would probably make more sense to have called it !await) thing isn't something I involved rollup in. It only really made sense to me on the backend.
I noticed a lot of the import architecture is already async. On the backend it's actually nonsensical that you have to await await and await again on importing sometimes. I think the reason it doesn't cause me too many headaches is because I use it sparingly only when it makes sense.
It's basically a no contest that I need it, module imports will already be async for me as I compile on demand with the loader for convenience and that often uses async stuff but also if I have some relatively commonly used module deep down in the dependency tree then all its dependants end up having to be async, you suddenly have 5% making the other 80% have to await their static imports anyway. Even dynamic imports you end up with await await import(); My kludge solutions aren't making it pleasure where I might want to export a promise.
It reached a point I started to think there should be an await_until_unawaitable keyword. It might also be worth considering the main reason for not having top level await typically owes more in many implementations to the way async is normally implemented traditionally depending on a special scope such as being inside a generator rather than out of any real design choice. My custom implementations all have had no top level only because the mechanisms used to implement them happened to not lend well to that nor have access to bootstrap the default scope. Often it made no natural sense for scripts that were entirely async based where the whole existence of the initial scope would virtually always be to switch into an async scope.
If people are going to look for problems I think it needs to be technical problems that takes precedence. Not making out like PICNIC is our own personal responsibility. I don't have a problem to the concept but people have implemented these things wrong before because of getting caught up in subjective rather than objective concerns and not focusing on the technical problem which also tends to include over looking less common but actually valid use cases so it needs scrutiny. I don't think this being a footgun is such a problem but other well known footgun's might be such as rushing an incomplete or ill considered specification out of the door that then can't change while relying on blind procedure to assume correctness over expert consultation, prototyping, experimentation, beta testing, etc.
I think there might be a small mind the gap scope for confusion around state that might exist with any import returning something like a persistent connection. That is, you import the thing to get the resource, then something else is allowed to happen, then the resource enters an oh no state such as dead connection then when you use if from imported the first time expecting it to be in an ok state it might not be.