Skip to content

Instantly share code, notes, and snippets.

@joepie91
Last active November 15, 2024 13:58
Show Gist options
  • Save joepie91/bca2fda868c1e8b2c2caf76af7dfcad3 to your computer and use it in GitHub Desktop.
Save joepie91/bca2fda868c1e8b2c2caf76af7dfcad3 to your computer and use it in GitHub Desktop.
ES Modules are terrible, actually

ES Modules are terrible, actually

This post was adapted from an earlier Twitter thread.

It's incredible how many collective developer hours have been wasted on pushing through the turd that is ES Modules (often mistakenly called "ES6 Modules"). Causing a big ecosystem divide and massive tooling support issues, for... well, no reason, really. There are no actual advantages to it. At all.

It looks shiny and new and some libraries use it in their documentation without any explanation, so people assume that it's the new thing that must be used. And then I end up having to explain to them why, unlike CommonJS, it doesn't actually work everywhere yet, and may never do so. For example, you can't import ESM modules from a CommonJS file! (Update: I've released a module that works around this issue.)

And then there's Rollup, which apparently requires ESM to be used, at least to get things like treeshaking. Which then makes people believe that treeshaking is not possible with CommonJS modules. Well, it is - Rollup just chose not to support it.

And then there's Babel, which tried to transpile import/export to require/module.exports, sidestepping the ongoing effort of standardizing the module semantics for ESM, causing broken imports and require("foo").default nonsense and spec design issues all over the place.

And then people go "but you can use ESM in browsers without a build step!", apparently not realizing that that is an utterly useless feature because loading a full dependency tree over the network would be unreasonably and unavoidably slow - you'd need as many roundtrips as there are levels of depth in your dependency tree - and so you need some kind of build step anyway, eliminating this entire supposed benefit.

And then people go "well you can statically analyze it better!", apparently not realizing that ESM doesn't actually change any of the JS semantics other than the import/export syntax, and that the import/export statements are equally analyzable as top-level require/module.exports.

"But in CommonJS you can use those elsewhere too, and that breaks static analyzers!", I hear you say. Well, yes, absolutely. But that is inherent in dynamic imports, which by the way, ESM also supports with its dynamic import() syntax. So it doesn't solve that either! Any static analyzer still needs to deal with the case of dynamic imports somehow - it's just rearranging deck chairs on the Titanic.

And then, people go "but now we at least have a standard module system!", apparently not realizing that CommonJS was literally that, the result of an attempt to standardize the various competing module systems in JS. Which, against all odds, actually succeeded!

... and then promptly got destroyed by ESM, which reintroduced a split and all sorts of incompatibility in the ecosystem, rather than just importing some updated variant of CommonJS into the language specification, which would have sidestepped almost all of these issues.

And while the initial CommonJS standardization effort succeeded due to none of the competing module systems being in particularly widespread use yet, CommonJS is so ubiquitous in Javascript-land nowadays that it will never fully go away. Which means that runtimes will forever have to keep supporting two module systems, and developers will forever be paying the cost of the interoperability issues between them.

But it's the future!

Is it really? The vast majority of people who believe they're currently using ESM, aren't even actually doing so - they're feeding their entire codebase through Babel, which deftly converts all of those snazzy import and export statements back into CommonJS syntax. Which works. So what's the point of the new module system again, if it all works with CommonJS anyway?

And it gets worse; import and export are designed as special-cased statements. Aside from the obvious problem of needing to learn a special syntax (which doesn't quite work like object destructuring) instead of reusing core language concepts, this is also a downgrade from CommonJS' require, which is a first-class expression due to just being a function call.

That might sound irrelevant on the face of it, but it has very real consequences. For example, the following pattern is simply not possible with ESM:

const someInitializedModule = require("module-name")(someOptions);

Or how about this one? Also no longer possible:

const app = express();
// ...
app.use("/users", require("./routers/users"));

Having language features available as a first-class expression is one of the most desirable properties in language design; yet for some completely unclear reason, ESM proponents decided to remove that property. There's just no way anymore to directly combine an import statement with some other JS syntax, whether or not the module path is statically specified.

The only way around this is with await import, which would break the supposed static analyzer benefits, only work in async contexts, and even then require weird hacks with parentheses to make it work correctly.

It also means that you now need to make a choice: do you want to be able to use ESM-only dependencies, or do you want to have access to patterns like the above that help you keep your codebase maintainable? ESM or maintainability, your choice!

So, congratulations, ESM proponents. You've destroyed a successful userland specification, wasted many (hundreds of?) thousands of hours of collective developer time, many hours of my own personal unpaid time trying to support people with the fallout, and created ecosystem fragmentation that will never go away, in exchange for... fuck all.

This is a disaster, and the only remaining way I see to fix it is to stop trying to make ESM happen, and deprecate it in favour of some variant of CommonJS modules being absorbed into the spec. It's not too late yet; but at some point it will be.

@ClickClickDerk
Copy link

IDK this might be old but i learned js5 like 12 years ago and took a break the past like 5 years just coding little projects for myself. I'm here today cause i dont understand modules and find them to be dumb. you can break up you js code to files that pages depend on it. I also taught myself to code so i went through one hell of a learning curve. i learned html, css, then js (client-side) before ever looking at how a server side language works and i remember because js is a loosely type language it made learning php or python more challanging and vice versa for the guys that knew strict server side programming didnt understand js to well. looking at it now es6 looks like the server side guys are turning the loosely typed language into be more readable as they understand how server side works. looking at all these js files and how AI is programming im like not every file has to be 1-2kb or less and have a ton of modules. i wrote free movies site back in the day must of had over 200,000 lines of js and it would load a bit slow on a phone but a desktop or laptop no problem. its 10 years later and everything is 100x faster. break up your files, lazy load, use ajax and you code is blazing fast today that way but now im hitting walls left and right with modules. if babel is really just turning es6 into older js syntax then that doesnt make sense. i look at babel as owning everyones code and going to strong arm people into making js and babel the standard. eww javascript is a beautiful language and es6 is ruining it. imo lol

@cmawhorter
Copy link

here i stand sit today in hour three of trying to get a complicated esm project packaged. this is -- not exaggerating -- the 10th time i've been here in the last several years with various projects.

this time around, i've tried doing everything right: it's a fresh project, i use esm deps AFAIK (sub-deps. who knows), i'm building it in node22 with a type: module in package.json. the only caveat to is that i'm using typescript, but i've followed all those guides people point to that exclaim esm is easy and it's time to use it.

from my experience, esm only works in the lab.

It's incredible how many collective developer hours have been wasted on pushing through the turd that is ES Modules

though i don't agree esm is a turd, OP is absolutely correct about that opening line. i remember filling out a dev survey about the possibility of esm long ago and begging in my responses to not do exactly what has been done.

so here we all are. and here i am. again. considering abandoning esm and using cjs because it just fucking works and who has the time for all this bullshit.

i'll see you all again in six months.

@guest271314
Copy link

@cmawhorter

i'm building it in node22 with a type: module in package.json. the only caveat to is that i'm using typescript.

That's one way of doing things.

Another way is to just use node --experimental-default-type=module --experimental-network-imports, static import and dynamic import() without any package.json file at all.

Yet another way is to use Deno, which support TypeScript and Import Maps out of the box. Again, no package.json involved, unless you want package.json involved.

Good luck!

@cmawhorter
Copy link

cmawhorter commented Jul 25, 2024 via email

@dboreham
Copy link

The last comments above I think illustrate a larger problem which is that tooling is "fixed" for the node.js case (cli process running on Linux) but the same tooling is also used to build browser-hosted applications. In my experience the tooling isn't fixed for those use cases.

@jensbodal
Copy link

I wouldn't call using experimental flags a "fix", just progress towards the goal. Fact is it's 4 years after this post was created and it's not easy at all to setup ESM compatible packages in a ubiquitous consistent way. At least for the frontend and my uses cases I can just use Vite for the backend. Unfortunately the Vite+NestJS options haven't worked out so far for us so still stuck on CJS for our NestJs+Express/Fastify backends.

@Rush
Copy link

Rush commented Jul 25, 2024

here i stand sit today in hour three of trying to get a complicated esm project packaged. this is -- not exaggerating -- the 10th time i've been here in the last several years with various projects.

this time around, i've tried doing everything right: it's a fresh project, i use esm deps AFAIK (sub-deps. who knows), i'm building it in node22 with a type: module in package.json. the only caveat to is that i'm using typescript, but i've followed all those guides people point to that exclaim esm is easy and it's time to use it.

from my experience, esm only works in the lab.

It's incredible how many collective developer hours have been wasted on pushing through the turd that is ES Modules

though i don't agree esm is a turd, OP is absolutely correct about that opening line. i remember filling out a dev survey about the possibility of esm long ago and begging in my responses to not do exactly what has been done.

so here we all are. and here i am. again. considering abandoning esm and using cjs because it just fucking works and who has the time for all this bullshit.

i'll see you all again in six months.

To me, the worst part of all of this is that tens of thousands of developers will take all of this for granted. Thinking that all of that has to be hard and one needs PhD in tooling. Ridiculous.

@mk-pmb
Copy link

mk-pmb commented Jul 25, 2024

Everyone complaining please provide links to your projects so that people interested in improving our situation have lots of test material.

@csvan
Copy link

csvan commented Jul 25, 2024

Everyone complaining please provide links to your projects so that people interested in improving our situation have lots of test material.

I would wager the vast majority of people "complaining" here - including me - work in corporate environments with closed sources.

@guest271314
Copy link

I'm not tied to Node.js exclusively.

If there is a Node.js-specific package I'm interested in, I immediately bundle to Ecmascript Modules with bun build or deno bundle, or deno_emit. No more CommonJS and require(). That's it.

No need for esbuild or webpack. Just use multiple JavaScript runtimes that do what Node.js can't or doesn't do, then use the bundeld code in Node.js world. If you must only use Node.js.

There are dozens of JavaScript engines and runtimes circa 2024. Why would any JavaScript programmer, developer, or hacker only use one?

@JamesAlp
Copy link

here i stand sit today in hour three of trying to get a complicated esm project packaged. this is -- not exaggerating -- the 10th time i've been here in the last several years with various projects.
this time around, i've tried doing everything right: it's a fresh project, i use esm deps AFAIK (sub-deps. who knows), i'm building it in node22 with a type: module in package.json. the only caveat to is that i'm using typescript, but i've followed all those guides people point to that exclaim esm is easy and it's time to use it.
from my experience, esm only works in the lab.

It's incredible how many collective developer hours have been wasted on pushing through the turd that is ES Modules

though i don't agree esm is a turd, OP is absolutely correct about that opening line. i remember filling out a dev survey about the possibility of esm long ago and begging in my responses to not do exactly what has been done.
so here we all are. and here i am. again. considering abandoning esm and using cjs because it just fucking works and who has the time for all this bullshit.
i'll see you all again in six months.

To me, the worst part of all of this is that tens of thousands of developers will take all of this for granted. Thinking that all of that has to be hard and one needs PhD in tooling. Ridiculous.

I'm not sure if that was meant to be a dig or what, but regardless I'd be interested to hear what your tooling is that makes using ESM easier if you're willing to share. I fully moved back to just CJS because I didn't really want to keep getting a headache trying to figure it out.

@mk-pmb
Copy link

mk-pmb commented Jul 26, 2024

I would wager the vast majority of people "complaining" here - including me - work in corporate environments with closed sources.

It may still be possible to make a minimum failing example with very similar structure. Even if management doesn't consider it worth work time, someone may opt to do it in their leisure time just for the hopes of a less annoying job experience soon. If management fears that even a minimum failing example would be too much disclosure, the hopes of a more efficient future workflow may convince them.

@JamesAlp
Copy link

I would wager the vast majority of people "complaining" here - including me - work in corporate environments with closed sources.

It may still be possible to make a minimum failing example with very similar structure. Even if management doesn't consider it worth work time, someone may opt to do it in their leisure time just for the hopes of a less annoying job experience soon. If management fears that even a minimum failing example would be too much disclosure, the hopes of a more efficient future workflow may convince them.

This is a pretty naïve way of thinking. Upper management at a company almost never care about the particular way a technology works, they just care if it works or not. Those managers aren't thinking of how they can improve the technology they don't even know the name of when they go to work every day, they're thinking "how do I get these numbers to go up?" and anything else better be very directly related to that topic or they don't care at all.

@guest271314
Copy link

Wow. Sounds like a hostile work environment. Stifling. No thank, you.

On the other hand, professional programmers should know how to make things work, on their own. So it's not matter of how to do something. It's solely a matter of what people that cut the checks want.

I read above about confusion re the former. And resignation as to the latter.

You have to want to understand how something works in order to make that process work.

@mk-pmb
Copy link

mk-pmb commented Jul 27, 2024

Yeah. If a company's programmers' efficiency and/or morale suffers from ESM incompatibilities, and management impedes the release of a minimum failing example project, their problem is no longer about ESM.

@JamesAlp
Copy link

That's just the thing though. Does a company suffer from lack of esm support? I would wager most don't bother with esm.

@iambumblehead
Copy link

Common Lisp, Perl and Python were similarly marred with "bad decisions" after becoming popular, what a coincidence!

@rubengmurray
Copy link

chai-as-promised(@8.0.0) another one that creates dreaded compatibility errors now. There really should be some railguards in-place that an installation of ESM dependency is prevented in a non-ESM project. Such a waste of time.

@guest271314
Copy link

There really should be some railguards in-place that an installation of ESM dependency is prevented in a non-ESM project.

Read the source code?

https://www.npmjs.com/package/chai-as-promised?activeTab=code at /lib/chai-as-promised.js

import * as checkErrorDefault from 'check-error';

@rubengmurray
Copy link

rubengmurray commented Aug 4, 2024

Read the reality?

Running this in a CommonJS project:git:(master) npm i chai-as-promised installs 8.0.0 and the installation process does not complain. Despite the fact it won't work. How many hours would have been saved if this check on npm i was present?

If you're going to create a system that has incompatibilities follow the golden rule: fail fast, and fail early.

@guest271314
Copy link

I don't think you understood what I proposed.

Read the source code of third-party code before installing.

It took me a few seconds.

@guest271314
Copy link

and the installation process does not complain.

Why would it?

Ecmascript Modules are the standard module loader system for JavaScript, as specified in ECMA-262.

The installer script, npm, doesn't know you are running CommonJS exclusively. Only you know that.

Nothing is stopping you from compiling the NPM modules to a single executable with deno compile or bun build --compile, then module loaders don't matter at all.

@rubengmurray
Copy link

You've solved it. Cheers @guest271314. Close the gist.

@guest271314
Copy link

There's really no problem statement.

There are tools available to do whatever you want.

@guest271314
Copy link

Using import and require() together

In Bun, you can use import or require in the same file—they both work, all the time.

import { stuff } from "./my-commonjs.cjs";
import Stuff from "./my-commonjs.cjs";
const myStuff = require("./my-commonjs.cjs");

@WilliamParker11
Copy link

WilliamParker11 commented Aug 14, 2024

Thanks for the info. For those in need of a book report, this https://academized.com/book-report-writing-service service offers professional writing help, ensuring that the reports are original, comprehensive, and meet academic standards. The platform connects students with experienced writers who deliver quality work.

@pedrolzoliveira
Copy link

I hate that you can't stub things properly with esm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment