Skip to content

Instantly share code, notes, and snippets.

@chrisdickinson

chrisdickinson/esm.md Secret

Created November 10, 2017 01:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save chrisdickinson/48821f3dd8d4b235a3fb33a666db83ab to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save chrisdickinson/48821f3dd8d4b235a3fb33a666db83ab to your computer and use it in GitHub Desktop.
Download ZIP
Raw
esm.md

esm, hurk

walking through the codebase

  • list indention means "I stepped into the code here"
  • TKTKTK means "to come", borrowed from isaac, I have no idea what it really means
  • I've broken these down into sections, the toplevel list items are "entry points" into a file
    • from there, list items in order mean "this executed in this order"
  • another note for background: anytime you see something like "FrobnicatorWrap", that is a strong indication that there's a binding from JS to C++ backing the object. Word to the wise!

Entry Point

node.cc - entry point

  • config_experimental_modules
  • config_userland_loader node_config.cc - InitConfig - configuration
  • config_experimental_modules -> JS-visible experimentalModules
  • config_userland_loader -> JS-visible userLoader
  • following experimentalModules:
    • lib/internal/bootstrap_node.js L103: only used in bootstrap to issue a warning about how truly experimental it is
    • lib/module.js
      • L451, Module._load -- if we don't have an ESMLoader yet, create one
      • Loader comes from internal/loader/Loader.js (oh lord capcase)
      • We use loader to load the userland loader hooks
      • then we continue to load using getURLFromFilePath
        • this uses the WHATWG URL to create a file:// url
      • we call Loader#import
      • ∅ _load exits

Loader

exports a Loader class, which inherits from null (a very bradley thing to do)

Loader entry points:

  • constructor: covered in the type def
  • hook({resolve, dynamicInstantiate}): sets a custom resolver and dynamicInstantiate
  • import(specifier, parentURL = this.base): this seems load-bearing, if you'll excuse the pun!
    • await this.getModuleJob(specifier, parentURL) returns a ModuleJob
      • calls this.resolve (usually ModuleRequest.resolve) L101
      • checks to see if we've got an outstanding ModuleJob in our moduleMap
        • if so, return it
        • ∅ getModuleJob exits (great job all)
      • resolve returns a url and a format
      • if the format is mundane (i.e., "not dynamic") pick the loader instance off of ModuleRequest.loaders
      • otherwise create a dynamic loader instance
      • pass the loader instance to ModuleJob
      • END result, returns a ModuleJob
    • gets a module from job.run()
    • returns module.namespace()

module request

entry points:

  • static loaders: loaders are provided for:
    • esm -- uses new ModuleWrap (see module wrap)
    • cjs -- uses createDynamicModule (from moduleWrap) (see dynamic loads)
    • builtin -- ditto
    • addon -- ditto
    • json -- surprisingly, ditto
  • static resolve: a Resolver
    • checks for builtins,
    • searches for the file,
    • checks the extension if it exists,
      • throws an error if it doesn't

module job

Entry points

  • constructor
    • constructing a ModuleJob immediately attempts to link all dependend modules (via ModuleWrap#link (see link in module wrap))
  • run
    • calls this.instantiate()
      • visits all modules in dependency graph, adding them to a set
      • once all modules have been visited, calls ModuleWrap#instantiate (see instantiate in module wrap)
        • This instantiates all modules in the graph!
      • then we do some bookkeeping by setting all of our dependency's instantiated props to a resolved promise
    • calls evaluate in module wrap

module wrap

The C++ binding to V8.

  • new ModuleWrap(source, url) - compile a module
  • ModuleWrap#link(resolver : Function)
    • Iterate over all direct dependencies of the current module
    • for each dependency, call anon async fn on lib/internal/loader/ModuleJob.js#L33
      • this recursively creates ModuleJob objects, linking them
      • internally we populate a map of strings to promises for modules (resolve_cache_)
  • ModuleWrap#instantiate()
    • calls V8's InstantiateModule with ResolveCallback
      • resolve callback is called by v8, and it expects a module to be returned
      • this is how we look up modules!
  • ModuleWrap#evaluate()
    • This actually executes the module!
  • ModuleWrap#namespace()
    • This returns the exported namespace of the module.

dynamic loads

Anything that's not ESM is dynamic. There are specializations for cjs, builtin, json, and addon modules (ref static loaders in module request). These specializations use dynamic modules!

See createDynamicModule in lib/internal/loader/ModuleWrap.js.

  • We perform a magic trick.
  • First we create a module that exports an executor variable
  • The module's completion value (that is, the last value evaluted) is a function returning an object with a setter for the executor variable, and a reflect object with getters and setters for individual exported names.
  • We immediately instantiate this facade module.
  • Then we evaluate it, returning the function exposing the executor and exports setters.
  • THEN we create another module that imports "executor" and other exports from ""
    • it also exports those variables
    • and if "executor" is a function, we call it
      • calling executor will mutate those variables so we end up getting the right values
  • We link this fronting module with a resolver that always returns our facade.
  • We instantiate it and send it out into the world.

types

class Loader

Optionally takes base.

  • moduleMap : ModuleMap: a Map of strings to ModuleJobs
  • base : String: defaults to the file:// url for the CWD
  • resolver : Resolver: a Resolver returning a promise for a url & format
    • if userLoader is not set, this will be ModuleRequest.resolve
  • dynamicInstantiate : Function: for "dynamic" loads

class ModuleJob

Takes a loader, url, and moduleProvider (a ModuleProvider).

  • loader : Loader: The loader that created the job via getModuleJob
  • error : null:
  • hadError : Boolean:
  • modulePromise : Promise<{module : ModuleWrap, reflect : null | ???}>:
  • module : undefined | ModuleWrap: Populated with a ModuleWrap once linked.
  • reflect : undefined | ????: Populated once linked. Unclear what reflect is for.
  • linked : Promise<Array<ModuleJob>>: a promise for the graph of modules depended upon by the current module
  • instantiated : undefined | Promise<>:

fn ModuleProvider

ModuleProvider ::= (
  url : String
) -> Promise<{
  module : ModuleWrap,
  reflect : null | ???
}>

fn Resolver

Resolver ::= (
  specifier : String,
  parentURL : String,
  defaultResolve : Resolver = ModuleRequest.resolve
) -> Promise<{
  url: String,
  format: ValidFormt
}>

summarize (tl;dr)

  • one major cutting point in Module._load
  • provide support for .coffee et al by adding a custom loader
    • no support for doing this at runtime, which seems a little prescriptive
  • dynamic imports support cjs, json, addons, etc
  • in ESM as implemented, require is already undefined
    • we can, in fact, reserve nodejs as a special module name
    • we should be able to "import {require} from nodejs" using the facade pattern
@bmeck
Copy link

bmeck commented Jan 13, 2018

no support for doing this at runtime, which seems a little prescriptive

This is by design due to limitations around conflicts of mutating the loader system at runtime. It also mimics problems with replacing service workers etc. in the browser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment