debugging lit-node + esm [UNLISTED]

README.md

Literate programming is a software development practice which treats source code like written material intended for humans instead of instructions intended for execution by computers. The most popular modern approach is to embed the source code in Markdown code blocks, surrounded by written explanatory text, comments, and other documentation, and then parse the Markdown before executing or compiling in order to extract only the code portions. In Node.js, one of the cleanest ways to do this is with the lit-node module, which works by adding a hook to the global require() function which is automatically added to the Node.js environment in order to load CommonJS modules, after which Markdown documents can be transpiled on the fly. Unfortunately lit-node seems to conflict with the esm module loader, which means it is not currently possible to use both literate style and ESM module format simultaneously from the Node CLI directly. It can still be done with a more elaborate setup based on lit.sh or blaze, but this is a fairly obvious combination and it would be better for the module loading tools to cooperate.

To reproduce, start by cloning this repository and installing both modules.

# install modules
$ npm install

You can now execute JavaScript code from Markdown documents, including this very README!

For example, here's some JavaScript:

// log a message
console.log('hello world')

Note that lit-node only executes code which is explicitly marked as javascript or js after the triple backticks. The rest of the code blocks in this document are bash, and are thus ignored.

You'll see the message above logged to the terminal if you run Node with the require() hook supplied by lit-node and specify this Markdown document as the script to be executed:

# execute the JavaScript code in this README
$ node --require lit-node/register README.md

This doesn't work with esm, however. The included esm.md file should be executable in much the same way, but it fails because it contains an ESM import that the esm module doesn't resolve.

# test ES6 module loading

$ node --require esm --require lit-node/register esm.md
# or
$ npm run esm

The specific error message suggests that even though the esm module has been loaded, Node is still confused by ESM import syntax by the time it gets to the lit-node hook:

> [email protected] esm /path
> node --require esm --require lit-node/register esm.md

/path/esm.md:4
import { ok } from 'assert'
       ^

SyntaxError: Unexpected token {
    at new Script (vm.js:83:7)
    at Object.<anonymous> (/path/node_modules/esm/esm.js:1)
    at Object.o._compile (/path/node_modules/esm/esm.js:1)
    at Object.o._compile (/path/node_modules/esm/esm.js:1)
    at Object.require.extensions..md (/path/node_modules/lit-node/register.js:7:16)
    at /path/node_modules/esm/esm.js:1
    at /path/node_modules/esm/esm.js:1
    at /path/node_modules/esm/esm.js:1
    at su (/path/node_modules/esm/esm.js:1)
    at Su (/path/node_modules/esm/esm.js:1)

Now, the test is not failing, exactly; rather, the entire testing script is crashing because the assert module cannot be imported with ESM syntax.

For convenience, this repository also includes a test for importing CommonJS modules (which already works just fine), and npm run test runs both the CommonJS and ESM tests.

# test CommonJS module loading
$ npm run cjs

# test both CommonJS and ESM module loading
$ npm run test

This problem will be fixed when the npm run test command exits without any errors.

See also:

• an issue with esm about this interaction
• a distinct but related issue with lit-node that was contributing to one possible version of the crash

cjs.md

test CommonJS module loader

const ok = require('assert').ok
ok(true, 'failed to load module with CommonJS require() function')

esm.md

test ES6 module loader

import { ok } from 'assert'
ok(true, 'failed to load module with ESM import statement')

package.json

{
  "name": "lit-node-esm-debug",
  "version": "0.0.1",
  "description": "demonstration to help with debugging module loading problems",
  "main": null,
  "dependencies": {
    "esm": "^3.0.84",
    "lit-node": "^0.1.1"
  },
  "scripts": {
    "test": "npm run cjs && npm run esm",
    "cjs": "node --require lit-node/register cjs.md",
    "esm": "node --require esm --require lit-node/register esm.md"
  },
  "keywords": [
    "debug",
    "modules",
    "literate",
    "programming"
  ],
  "author": "Vijith Assar",
  "license": "UNLICENSED"
}