Skip to content

Instantly share code, notes, and snippets.

@joepie91
Last active February 25, 2025 06:17
Show Gist options
  • Save joepie91/bca2fda868c1e8b2c2caf76af7dfcad3 to your computer and use it in GitHub Desktop.
Save joepie91/bca2fda868c1e8b2c2caf76af7dfcad3 to your computer and use it in GitHub Desktop.
ES Modules are terrible, actually

ES Modules are terrible, actually

This post was adapted from an earlier Twitter thread.

It's incredible how many collective developer hours have been wasted on pushing through the turd that is ES Modules (often mistakenly called "ES6 Modules"). Causing a big ecosystem divide and massive tooling support issues, for... well, no reason, really. There are no actual advantages to it. At all.

It looks shiny and new and some libraries use it in their documentation without any explanation, so people assume that it's the new thing that must be used. And then I end up having to explain to them why, unlike CommonJS, it doesn't actually work everywhere yet, and may never do so. For example, you can't import ESM modules from a CommonJS file! (Update: I've released a module that works around this issue.)

And then there's Rollup, which apparently requires ESM to be used, at least to get things like treeshaking. Which then makes people believe that treeshaking is not possible with CommonJS modules. Well, it is - Rollup just chose not to support it.

And then there's Babel, which tried to transpile import/export to require/module.exports, sidestepping the ongoing effort of standardizing the module semantics for ESM, causing broken imports and require("foo").default nonsense and spec design issues all over the place.

And then people go "but you can use ESM in browsers without a build step!", apparently not realizing that that is an utterly useless feature because loading a full dependency tree over the network would be unreasonably and unavoidably slow - you'd need as many roundtrips as there are levels of depth in your dependency tree - and so you need some kind of build step anyway, eliminating this entire supposed benefit.

And then people go "well you can statically analyze it better!", apparently not realizing that ESM doesn't actually change any of the JS semantics other than the import/export syntax, and that the import/export statements are equally analyzable as top-level require/module.exports.

"But in CommonJS you can use those elsewhere too, and that breaks static analyzers!", I hear you say. Well, yes, absolutely. But that is inherent in dynamic imports, which by the way, ESM also supports with its dynamic import() syntax. So it doesn't solve that either! Any static analyzer still needs to deal with the case of dynamic imports somehow - it's just rearranging deck chairs on the Titanic.

And then, people go "but now we at least have a standard module system!", apparently not realizing that CommonJS was literally that, the result of an attempt to standardize the various competing module systems in JS. Which, against all odds, actually succeeded!

... and then promptly got destroyed by ESM, which reintroduced a split and all sorts of incompatibility in the ecosystem, rather than just importing some updated variant of CommonJS into the language specification, which would have sidestepped almost all of these issues.

And while the initial CommonJS standardization effort succeeded due to none of the competing module systems being in particularly widespread use yet, CommonJS is so ubiquitous in Javascript-land nowadays that it will never fully go away. Which means that runtimes will forever have to keep supporting two module systems, and developers will forever be paying the cost of the interoperability issues between them.

But it's the future!

Is it really? The vast majority of people who believe they're currently using ESM, aren't even actually doing so - they're feeding their entire codebase through Babel, which deftly converts all of those snazzy import and export statements back into CommonJS syntax. Which works. So what's the point of the new module system again, if it all works with CommonJS anyway?

And it gets worse; import and export are designed as special-cased statements. Aside from the obvious problem of needing to learn a special syntax (which doesn't quite work like object destructuring) instead of reusing core language concepts, this is also a downgrade from CommonJS' require, which is a first-class expression due to just being a function call.

That might sound irrelevant on the face of it, but it has very real consequences. For example, the following pattern is simply not possible with ESM:

const someInitializedModule = require("module-name")(someOptions);

Or how about this one? Also no longer possible:

const app = express();
// ...
app.use("/users", require("./routers/users"));

Having language features available as a first-class expression is one of the most desirable properties in language design; yet for some completely unclear reason, ESM proponents decided to remove that property. There's just no way anymore to directly combine an import statement with some other JS syntax, whether or not the module path is statically specified.

The only way around this is with await import, which would break the supposed static analyzer benefits, only work in async contexts, and even then require weird hacks with parentheses to make it work correctly.

It also means that you now need to make a choice: do you want to be able to use ESM-only dependencies, or do you want to have access to patterns like the above that help you keep your codebase maintainable? ESM or maintainability, your choice!

So, congratulations, ESM proponents. You've destroyed a successful userland specification, wasted many (hundreds of?) thousands of hours of collective developer time, many hours of my own personal unpaid time trying to support people with the fallout, and created ecosystem fragmentation that will never go away, in exchange for... fuck all.

This is a disaster, and the only remaining way I see to fix it is to stop trying to make ESM happen, and deprecate it in favour of some variant of CommonJS modules being absorbed into the spec. It's not too late yet; but at some point it will be.

@iambumblehead
Copy link

Common Lisp, Perl and Python were similarly marred with "bad decisions" after becoming popular, what a coincidence!

@rubengmurray
Copy link

chai-as-promised(@8.0.0) another one that creates dreaded compatibility errors now. There really should be some railguards in-place that an installation of ESM dependency is prevented in a non-ESM project. Such a waste of time.

@guest271314
Copy link

There really should be some railguards in-place that an installation of ESM dependency is prevented in a non-ESM project.

Read the source code?

https://www.npmjs.com/package/chai-as-promised?activeTab=code at /lib/chai-as-promised.js

import * as checkErrorDefault from 'check-error';

@rubengmurray
Copy link

rubengmurray commented Aug 4, 2024

Read the reality?

Running this in a CommonJS project:git:(master) npm i chai-as-promised installs 8.0.0 and the installation process does not complain. Despite the fact it won't work. How many hours would have been saved if this check on npm i was present?

If you're going to create a system that has incompatibilities follow the golden rule: fail fast, and fail early.

@guest271314
Copy link

I don't think you understood what I proposed.

Read the source code of third-party code before installing.

It took me a few seconds.

@guest271314
Copy link

and the installation process does not complain.

Why would it?

Ecmascript Modules are the standard module loader system for JavaScript, as specified in ECMA-262.

The installer script, npm, doesn't know you are running CommonJS exclusively. Only you know that.

Nothing is stopping you from compiling the NPM modules to a single executable with deno compile or bun build --compile, then module loaders don't matter at all.

@rubengmurray
Copy link

You've solved it. Cheers @guest271314. Close the gist.

@guest271314
Copy link

There's really no problem statement.

There are tools available to do whatever you want.

@guest271314
Copy link

Using import and require() together

In Bun, you can use import or require in the same file—they both work, all the time.

import { stuff } from "./my-commonjs.cjs";
import Stuff from "./my-commonjs.cjs";
const myStuff = require("./my-commonjs.cjs");

@pedrolzoliveira
Copy link

I hate that you can't stub things properly with esm

@Kreijstal
Copy link

you know something?

I... agree...

there is however one thing that moves me to esm.
That is .. esm.sh
Yes, I can use any module of npm on the browser with simply an import... magic!

If you can show me how to use this with CJS, you will convert me completely.
You'd have to build something like esm.sh.

.. I HATE build steps.
Please be the guy that destroys ESM..

@michaeljnash
Copy link

michaeljnash commented Jan 12, 2025 via email

@Kreijstal
Copy link

So guys, wild idea... We are all a group of talented developers (I assume for most of you) What about we start a project to rival ESM that is fundamentally better and will have no problem becoming the new standard eventually? Or fork ESM code to something more... Sensible in certain areas? Crazier things have been done. Instead of complaining about it... Let's fix it?

On Sun, Jan 12, 2025, 8:41 AM Kreijstal @.> wrote: @.* commented on this gist. ------------------------------ you know something? I... agree... there is however one thing that moves me to esm. That is .. esm.sh Yes, I can use any module of npm on the browser with simply an import... magic! If you can show me how to use this with CJS, you will convert me completely. You'd have to build something like esm.sh. .. I HATE build steps. Please be the guy that destroys ESM.. — Reply to this email directly, view it on GitHub https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7dfcad3#gistcomment-5385507 or unsubscribe https://github.com/notifications/unsubscribe-auth/AVDLF4LUJKV4ZOMTU23LQND2KKEI7BFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTCMBSGAZDIMRVU52HE2LHM5SXFJTDOJSWC5DF . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

so esm.sh converts CJS modules to ESM modules, using transpilation, and converts paths to urls, we could do that with require, we require a full url path, and rewrite url, we would have CJS on the browser, a dream.

@guest271314
Copy link

So guys, wild idea... We are all a group of talented developers (I assume for most of you) What about we start a project to rival ESM that is fundamentally better and will have no problem becoming the new standard eventually? Or fork ESM code to something more... Sensible in certain areas? Crazier things have been done. Instead of complaining about it... Let's fix it?

Use Modules as defined by ECMA-262, use CommonJS. Use whatever tools in the JavaScript toolbox that suits completing the given task.

Right now I'm using node nightly, deno canary, bun canary, hermes, shermes, workerd, d8 (V8 shell), js (SpiderMonkey shell), llrt, qjs, tjs.

Do you think each of those JavaScript runtimes implements CommonJS or ECMA-262 Modules the same?

No.

node doesn't support network imports using CommonJS, obviously, nor for ECMA-262 Modules.

deno does support network import and import().

ECMA-262 has a big 'ole "or" in the specification with regard to host implentations and dynamic import(). deno throws for dynamic import() when the script is created in the running script and a raw string specifier is used.

bun doesn't support network imports for ECMA-262 import or import().

deno supports WICG Import Maps, providing the capability to dow essentially whatever you want with regard to specifiers and URL's or file references.

node and deno do not support WICG Import Maps.

Facebook's hermes and shermes don't support CommonJS or ECAM-262 that I am aware of.

bun can run CommonJS and ECMA-262 Modules at the same time in the same script.

deno bundle up to version 2 existed, and still exists for that executable version range, to compile CommonJS to ECMA-262 Modules, primarily with regard to Deno source code.

bun build compiles CommonJS to ECMA-262 Modules. That compiled module can then be worked on to run in node, deno, and bun environments. Requires writing code by hand in a great deal of instances. Possible.

Now, the details of that compilation, might not be 1:1 result. As is discussed in an esbuild issue about conversion from CommonJS to ECMA-262. That's correct. Then program by hand. Line by line. I've done it. Multiple times. Particularly in cases where the maintainer decides to write source code exclusively for Node.js. Then Node.js API's also have to be adjusted; specifiers changed occasionally to use node:. Import Maps are useful for that purpose, too.

My current interest is writing JavaScript runtime (and engine) agnostic source code. It's both interesting and provides challenges, to me.

My suggestion would be to use all of the available tools in the JavaScript toolbox. Without necessarily entertaining a preference for any specific tool.

If the goal really is to unify the programming language.

Nothing is stopping CommonJS and ECMA-262 Modules existing at the same time in the same code base. As long as there is no rancor among the stackeholders.

If you think CommonJS compared to ECMA-262 is an unbridgeable distance, or challenging to bridge, try writing the same code that read stdin and writes to stdout that can run in different JavaScript runtimes.

ECMA-262 doesn't specify I/O. Each engine and runtime implements reading stdin and writing to stdout differently, if at all.

@Kreijstal
Copy link

My current interest is writing JavaScript runtime (and engine) agnostic source code. It's both interesting and provides challenges, to me.

How do you support different runtimes.. JerryScript, quickjs, boa etc... Do you test in CI all of them? :D

@guest271314
Copy link

Yes. Manually, and programmatically. I have briefly tested JerryScript recently. I was looking at boa yesterday.

@Kreijstal
Copy link

Yes. Manually, and programmatically. I have briefly tested JerryScript recently. I was looking at boa yesterday.

I mean obviously if you want to support all runtimes the idea is transpilation, no need to split into files, have everything in one big file, then convert everything to ES3, no?
But here it's about CJS, how it doesn't work on the browsers like ESM.

I can always (import("https://esm.sh/[email protected]").then(function(confetti){confetti.default()})) on a site and it will work

@guest271314
Copy link

I've done it a few different ways. Sometimes manual, line by line substitution, writing is necessary.

Here's reading stdin to node, deno, and bun using the same script https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_host.js

/*
#!/usr/bin/env -S /home/user/bin/deno -A /home/user/bin/nm_host.js
#!/usr/bin/env -S /home/user/bin/node /home/user/bin/nm_host.js
#!/usr/bin/env -S /home/user/bin/bun run /home/user/bin/nm_host.js
*/
import * as process from "node:process";
const runtime = navigator.userAgent;
const buffer = new ArrayBuffer(0, { maxByteLength: 1024 ** 2 });
const view = new DataView(buffer);
const encoder = new TextEncoder();
// const { dirname, filename, url } = import.meta;

let readable, writable, exit; // args

if (runtime.startsWith("Deno")) {
  ({ readable } = Deno.stdin);
  ({ writable } = Deno.stdout);
  ({ exit } = Deno);
  // ({ args } = Deno);
}

if (runtime.startsWith("Node")) {
  readable = process.stdin;
  writable = new WritableStream({
    write(value) {
       process.stdout.write(value);
    }
  });
  ({ exit } = process);
  // ({ argv: args } = process);
}

if (runtime.startsWith("Bun")) {
  readable = Bun.file("/dev/stdin").stream();
  writable = new WritableStream({
    async write(value) {
      await Bun.write(Bun.stdout, value);
    },
  }, new CountQueuingStrategy({ highWaterMark: Infinity }));
  ({ exit } = process);
  // ({ argv: args } = Bun);
}

function encodeMessage(message) {
  return encoder.encode(JSON.stringify(message));
}

async function* getMessage() {
  let messageLength = 0;
  let readOffset = 0;
  for await (let message of readable) {
    if (buffer.byteLength === 0 && messageLength === 0) {
      buffer.resize(4);
      for (let i = 0; i < 4; i++) {
        view.setUint8(i, message[i]);
      }
      messageLength = view.getUint32(0, true);
      message = message.subarray(4);
      buffer.resize(0);
    }
    buffer.resize(buffer.byteLength + message.length);
    for (let i = 0; i < message.length; i++, readOffset++) {
      view.setUint8(readOffset, message[i]);
    }
    if (buffer.byteLength === messageLength) {
      yield new Uint8Array(buffer);
      messageLength = 0;
      readOffset = 0;
      buffer.resize(0);
    }
  }
}

async function sendMessage(message) {
  await new Blob([
    new Uint8Array(new Uint32Array([message.length]).buffer),
    message,
  ])
    .stream()
    .pipeTo(writable, { preventClose: true });
}

try {
  // await sendMessage(encodeMessage([{ dirname, filename, url }, ...args]));
  for await (const message of getMessage()) {
    await sendMessage(message);
  }
} catch (e) {
  sendMessage(encodeMessage(e.message));
  exit();
}

export {
  encodeMessage,
  exit,
  getMessage,
  readable,
  sendMessage,
  writable,
};

A while ago I bundled ts-ebml for the browser. There is a polyfill for require() https://github.com/legokichi/ts-ebml/blob/a7ec9ddd54c2ff7ae7c0d3656864d0388eda238b/lib/ts-ebml-min.js#L4C1-L5C46

// see requiring npm modules in the browser console <https://gist.github.com/mathisonian/c325dbe02ea4d6880c4e>
// usgae `const tsebml = require("ts-ebml");`

Bun compiles require() into ECMA-262 Modules to an appreciable degree. I don't think a 1:1 conversion is possible; see evanw/esbuild#3947. Somewhere in esbuild repository there's a sentence in an issue where the maintainer says something like conversion between CommonJS and ECMAScript Modules 1:1 is impossible.

It was far more line by line modification to adjust code that was written exclusively for Node.js API's, for example substituting Web Cryptography API for node:crypto - which cannot be polyfilled (see https://github.com/guest271314/webbundle, https://github.com/guest271314/wbn-sign-webcrypto), than dealing just with the module loader, here Build/rebuild wbn-bundle.js from webbundle-plugins/packages/rollup-plugin-webbundle/src/index.ts with bun.

For implementing WASI for Deno, Node.js, Bun, I had to remove the maintainer's Deno references, and use Node.js API's https://github.com/guest271314/deno-wasi.

Notice the different approaches, basically the opposite, between the two examples above.

See using bun to get npm to compile to a standalone executable using node's SEA, along with adding some require("module") in the head of the script https://gist.github.com/guest271314/c9543a19d8ccf72881355b27d0107551.

node -e 'const file="bun-npm-bundle.js";const fs=require("fs");fs.writeFileSync(file, "require=require(\"node:module\").createRequire(__filename);\n" + fs.readFileSync(file, "utf8").replace("#!/usr/bin/env node", ""));'

It depends. In general, manual modification even of code bundled with a bundler might be necessary. There's no roadmap for the experiments and tests I do. JavaScript programmers, from my observation, tend to develop preferences and brand loyalties, and stay in Node.js world. There are exceptions. Some Node.js contributors/maintainers/members, whatever have contributed to QuickJS NG and Deno, etc.

I suggest testing bun build or esbuild, or as you mention esm.sh if the idea to convert CommonJS to ECMAScript Modules.

@Kreijstal
Copy link

what do you mean crypto can't be polyfilled? It won't be constant time, but it can be polyfilled.

@guest271314
Copy link

@Kreijstal

Can't be polyfilled.

[AskJS] Why is the internal crypto module so difficult to bundle as a standalone portable script?

The crypto module is not entirely Javascript. Some of its implementation uses native code which only runs in the nodejs environment and some of its implementation uses nodejs internals (like the thread pool).

Use Web Cryptography API for wbn-sign-webcrypto, crypto-browserify throws for subtle, doesn't implement Ed25519 #786

@Kreijstal
Copy link

Kreijstal commented Jan 13, 2025

Can't be polyfilled.

just because implementation is lacking doesn't mean it can not be polyfilled.
https://github.com/paulmillr/noble-ed25519
it is not an universal truth, maybe it can't right now, because there is no implementation written. Not that it can't be done, can't be written.

@guest271314
Copy link

I said node:crypto can't be polyfilled. We have standardized Web Cryptography API. We have standardized ECMAScript Modules. Node.js is still using non-standard CommonJS. Now, does that mean implementers and programmers in the field should always follow specifications? No. WHATWG Fetch specification, the last time I checked, still has Request resolving to a Promise only when the body has been completely read. HTTP allows for response while the request is still being read. That is, full-duplex streaming using fetch() whatwg/fetch#1254. Node.js, Deno, and now Bun oven-sh/bun#7206 support full-duplex streaming with fetch(), as demonstrated here https://github.com/guest271314/native-messaging-nodejs/tree/full-duplex, here https://github.com/guest271314/native-messaging-deno/tree/fetch-duplex, and here https://github.com/oven-sh/bun/files/13400754/full_duplex_fetch_test.js.zip, although WHATWG Fetch doesn't spell out the mechanics.

@Kreijstal
Copy link

Kreijstal commented Jan 14, 2025

I said node:crypto can't be polyfilled.

What methods or funtions of node:crypto can't be polyfilled?

@guest271314
Copy link

This is the short answer

[AskJS] Why is the internal crypto module so difficult to bundle as a standalone portable script?

The crypto module is not entirely Javascript. Some of its implementation uses native code which only runs in the nodejs environment and some of its implementation uses nodejs internals (like the thread pool).

The long answer is I will have to go back over all of the questions I asked in different venues, and the code I ran in Deno and Bun when trying to write runtime agnostic JavaScript that just so happened to use node:crypto module, because the maintainers of wbn:sign decided to write code exclusively for Node.js. The path to the short answer included getting banned from a couple of repositories. But I finally figured out how to write the code without using node:crypto.

A brief summary, not in any particular order

But if you think my claim is erroneous, all you have to do is create a single script that runs the same in node, deno, and bun starting with this code

Here's my fork

node:crypto is like node:wasi that depends on uvwasi internally.

@Kreijstal
Copy link

But if you think my claim is erroneous, all you have to do is create a single script that runs the same in node, deno, and bun starting with this code

You want to connect to do TCP/IP from node, deno and bun? Also the browser, or just those 3? Do you need crypto for bare TCP/IP?

Right now I'm using node nightly, deno canary, bun canary, hermes, shermes, workerd, d8 (V8 shell), js (SpiderMonkey shell), llrt, qjs, tjs.

I discovered a runtime for you https://duktape.org/ also look at the page of Similar engines

@guest271314
Copy link

I don't need node:crypto, The authors of https://github.com/GoogleChromeLabs/telnet-client decided to write the code exclusively for Node.js. I expanded covereage to Deno and Bun, too, in part by substituting Web Cryptography API for nod:crypto after I found out node:crypto can't be polyfilled.

Yes, I've read about duktape. A list of JavaScript engines, runtimes, interpreters.

@pcj
Copy link

pcj commented Feb 24, 2025

The number of hours I've wasted on ESM vs CJS... And here I am again in 2025 trying to make jest work with ESM, for the nth time. Fuck me.

@Kreijstal
Copy link

The number of hours I've wasted on ESM vs CJS... And here I am again in 2025 trying to make jest work with ESM, for the nth time. Fuck me.

I use tape works everywhere

@Rush
Copy link

Rush commented Feb 24, 2025

The number of hours I've wasted on ESM vs CJS... And here I am again in 2025 trying to make jest work with ESM, for the nth time. Fuck me.

I am just glad to see the rant is alive. We are happily using CJS and staying productive. We use webpack to bundle both server and client code and mostly don't cure about all of this bullshit. Bundled code is all .cjs.

@guest271314
Copy link

Reads like corporate users of exclusively Node.js from 10 years ago.

It's great programmers have choices.

Will the next generation who fill current CommonJS users shoes in companies have the same commitment to CommonJS in 10 years?

Unlikely.

As unlikely as somebody predicting the creator of Node.js would create Deno that uses ECMAScript Modules, though still supports .cjs.

By the way, there's a whole world of JavaScript outside of Node.js itself: from QuickJS to Static Hermes to AssemblyScript to Bun which can run WASM diectly, C via TinyCC, bundle,and support .cjs and ECMAScript Modules, which in general the current and next genereation of JavaScript programmers use.

There's enough room for everybody and their philosophocal and technical preferences in JavaScript world.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment