esm, hurk

walking through the codebase

list indention means "I stepped into the code here"
TKTKTK means "to come", borrowed from isaac, I have no idea what it really means
I've broken these down into sections, the toplevel list items are "entry points" into a file
- from there, list items in order mean "this executed in this order"
another note for background: anytime you see something like "FrobnicatorWrap", that is a strong indication that there's a binding from JS to C++ backing the object. Word to the wise!

Entry Point

node.cc - entry point

config_experimental_modules
config_userland_loader node_config.cc - InitConfig - configuration
config_experimental_modules -> JS-visible experimentalModules
config_userland_loader -> JS-visible userLoader
following experimentalModules:
- lib/internal/bootstrap_node.js L103: only used in bootstrap to issue a warning about how truly experimental it is
- lib/module.js
  - L451, Module._load -- if we don't have an ESMLoader yet, create one
  - Loader comes from internal/loader/Loader.js (oh lord capcase)
    - see Loader
  - We use loader to load the userland loader hooks
    - see "hooks" in Loader
  - then we continue to load using getURLFromFilePath
    - this uses the WHATWG URL to create a file:// url
  - we call Loader#import
    - see Loader
  - ∅ _load exits

Loader

exports a Loader class, which inherits from null (a very bradley thing to do)

Loader entry points:

constructor: covered in the type def
hook({resolve, dynamicInstantiate}): sets a custom resolver and dynamicInstantiate
import(specifier, parentURL = this.base): this seems load-bearing, if you'll excuse the pun!
- await this.getModuleJob(specifier, parentURL) returns a ModuleJob
  - calls this.resolve (usually ModuleRequest.resolve) L101
    - see resolve in module request
  - checks to see if we've got an outstanding ModuleJob in our moduleMap
    - if so, return it
    - ∅ getModuleJob exits (great job all)
  - resolve returns a url and a format
  - if the format is mundane (i.e., "not dynamic") pick the loader instance off of ModuleRequest.loaders
    - see loaders in module request
  - otherwise create a dynamic loader instance
  - pass the loader instance to ModuleJob
    - see constructor in module job
  - END result, returns a ModuleJob
- gets a module from job.run()
  - see run in module job
- returns module.namespace()

module request

entry points:

static loaders: loaders are provided for:
- esm -- uses new ModuleWrap (see module wrap)
- cjs -- uses createDynamicModule (from moduleWrap) (see dynamic loads)
- builtin -- ditto
- addon -- ditto
- json -- surprisingly, ditto
static resolve: a Resolver
- checks for builtins,
- searches for the file,
- checks the extension if it exists,
  - throws an error if it doesn't

module job

Entry points

constructor
- constructing a ModuleJob immediately attempts to link all dependend modules (via ModuleWrap#link (see link in module wrap))
run
- calls this.instantiate()
  - visits all modules in dependency graph, adding them to a set
  - once all modules have been visited, calls ModuleWrap#instantiate (see instantiate in module wrap)
    - This instantiates all modules in the graph!
  - then we do some bookkeeping by setting all of our dependency's instantiated props to a resolved promise
- calls evaluate in module wrap

module wrap

The C++ binding to V8.

new ModuleWrap(source, url) - compile a module
ModuleWrap#link(resolver : Function)
- Iterate over all direct dependencies of the current module
- for each dependency, call anon async fn on lib/internal/loader/ModuleJob.js#L33
  - this recursively creates ModuleJob objects, linking them
  - internally we populate a map of strings to promises for modules (resolve_cache_)
ModuleWrap#instantiate()
- calls V8's InstantiateModule with ResolveCallback
  - resolve callback is called by v8, and it expects a module to be returned
  - this is how we look up modules!
ModuleWrap#evaluate()
- This actually executes the module!
ModuleWrap#namespace()
- This returns the exported namespace of the module.

dynamic loads

Anything that's not ESM is dynamic. There are specializations for cjs, builtin, json, and addon modules (ref static loaders in module request). These specializations use dynamic modules!

See createDynamicModule in lib/internal/loader/ModuleWrap.js.

We perform a magic trick.
First we create a module that exports an executor variable
The module's completion value (that is, the last value evaluted) is a function returning an object with a setter for the executor variable, and a reflect object with getters and setters for individual exported names.
We immediately instantiate this facade module.
Then we evaluate it, returning the function exposing the executor and exports setters.
THEN we create another module that imports "executor" and other exports from ""
- it also exports those variables
- and if "executor" is a function, we call it
  - calling executor will mutate those variables so we end up getting the right values
We link this fronting module with a resolver that always returns our facade.
We instantiate it and send it out into the world.

types

`class Loader`

Optionally takes base.

moduleMap : ModuleMap: a Map of strings to ModuleJobs
base : String: defaults to the file:// url for the CWD
resolver : Resolver: a Resolver returning a promise for a url & format
- if userLoader is not set, this will be ModuleRequest.resolve
dynamicInstantiate : Function: for "dynamic" loads

`class ModuleJob`

Takes a loader, url, and moduleProvider (a ModuleProvider).

loader : Loader: The loader that created the job via getModuleJob
error : null:
hadError : Boolean:
modulePromise : Promise<{module : ModuleWrap, reflect : null | ???}>:
module : undefined | ModuleWrap: Populated with a ModuleWrap once linked.
reflect : undefined | ????: Populated once linked. Unclear what reflect is for.
linked : Promise<Array<ModuleJob>>: a promise for the graph of modules depended upon by the current module
instantiated : undefined | Promise<>:

`fn ModuleProvider`

ModuleProvider ::= (
  url : String
) -> Promise<{
  module : ModuleWrap,
  reflect : null | ???
}>

`fn Resolver`

Resolver ::= (
  specifier : String,
  parentURL : String,
  defaultResolve : Resolver = ModuleRequest.resolve
) -> Promise<{
  url: String,
  format: ValidFormt
}>

summarize (tl;dr)

one major cutting point in Module._load
provide support for .coffee et al by adding a custom loader
- no support for doing this at runtime, which seems a little prescriptive
dynamic imports support cjs, json, addons, etc
in ESM as implemented, require is already undefined
- we can, in fact, reserve nodejs as a special module name
- we should be able to "import {require} from nodejs" using the facade pattern

chrisdickinson/esm.md Secret