Skip to content

Instantly share code, notes, and snippets.

@guest271314

guest271314/compiling_npm_to_standalone_executable.md

Last active December 1, 2024 05:03
Download ZIP
Compiling npm to a standalone executable: Which runtime can do this out of the box; node, deno, or bun?
Raw
compiling_npm_to_standalone_executable.md

Here's a modest project, just to see what happens: Try to convert npm to an Ecmascript Module and compile npm https://github.com/npm/cli to a single executable with deno compile, bun build, and node --experimental-sea-config combined with postject.

Node.js

npm is written in CommonJS.

node can't even handle compiling npm to a Single executable https://nodejs.org/api/single-executable-applications.html with

cp node npm
echo '{ "main": "node-v23/bin/npm", "output": "sea-prep.blob" }' > sea-config.json
node --experimental-sea-config sea-config.json 
./node_modules/postject/dist/cli.js npm  NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 --overwrite

due to

require('../lib/cli.js')(process)

require() in the injected main script is not the same as the require() ...

npm --help
(node:60422) Warning: Currently the require() provided to the main script embedded into single-executable applications only supports loading built-in modules.
To load a module from disk after the single executable application is launched, use require("module").createRequire().
Support for bundled module loading or virtual file systems are under discussions in https://github.com/nodejs/single-executable
(Use `node-npm --trace-warnings ...` to show where the warning was created)
node:internal/main/embedding:111
    throw new ERR_UNKNOWN_BUILTIN_MODULE(id);
    ^

Error [ERR_UNKNOWN_BUILTIN_MODULE]: No such built-in module: ../lib/cli.js
    at embedderRequire (node:internal/main/embedding:111:11)
    at node-v23/bin/npm:2:1
    at embedderRunCjs (node:internal/main/embedding:87:10) {
  code: 'ERR_UNKNOWN_BUILTIN_MODULE'
}

Node.js v23.0.0-nightly20241016019efe1453

Update

Bundle the npm-cli.js entry point to a CommonJS module

npm i npm # Using npm built with deno compile
bun build /home/user/bin/node_modules/npm/bin/npm-cli.js --target=node --format=cjs --outfile=bun-npm-bundle.js

  bun-npm-bundle.js  9.20 KB

[17ms] bundle 4 modules
cp node node-npm

Modify the CommonJS output by bun build to use replace the shebang and use createRequire(), see https://nodejs.org/api/single-executable-applications.html#requireid-in-the-injected-main-script-is-not-file-based.

node -e 'const file="bun-npm-bundle.js";const fs=require("fs");fs.writeFileSync(file, "require=require(\"node:module\").createRequire(__filename);\n" + fs.readFileSync(file, "utf8").replace("#!/usr/bin/env node", ""));'

cp node node-npm

echo '{ "main": "bun-npm-bundle.js", "output": "sea-prep.blob" }' > sea-config.json

node --experimental-sea-config sea-config.json 
Wrote single executable preparation blob to sea-prep.blob

node ./node_modules/postject/dist/cli.js node-npm  NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 --overwrite
Start injection of NODE_SEA_BLOB in node-npm...
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
💉 Injection done!

node-npm
(node:66302) ExperimentalWarning: Single executable application is an experimental feature and might change at any time
(Use `node-npm --trace-warnings ...` to show where the warning was created)
npm warn cli npm v10.9.0 does not support Node.js v23.0.0-nightly20241016019efe1453. This version of npm supports the following node versions: `^18.17.0 || >=20.5.0`. You can find the latest version at https://nodejs.org/.
npm <command>

Usage:

npm install        install all the dependencies in your project
npm install <foo>  add the <foo> dependency to your project
npm test           run this project's tests
npm run <foo>      run the script named <foo>
npm <command> -h   quick help on <command>
npm -l             display usage info for all commands
npm help <term>    search for help on <term>
npm help npm       more involved overview

All commands:

    access, adduser, audit, bugs, cache, ci, completion,
    config, dedupe, deprecate, diff, dist-tag, docs, doctor,
    edit, exec, explain, explore, find-dupes, fund, get, help,
    help-search, hook, init, install, install-ci-test,
    install-test, link, ll, login, logout, ls, org, outdated,
    owner, pack, ping, pkg, prefix, profile, prune, publish,
    query, rebuild, repo, restart, root, run-script, sbom,
    search, set, shrinkwrap, star, stars, start, stop, team,
    test, token, uninstall, unpublish, unstar, update, version,
    view, whoami

Specify configs in the ini-formatted file:
    /home/user/.npmrc
or on the command line via: npm <command> --key=value

More configuration info: npm help config
Configuration fields: npm help 7 config

npm@10.9.0 /home/user/bin/node_modules/npm

Deno

Deno has issues with the CommonJS source code of GitHub's (formerly Node.js's) npm, too, from the Node.js Nightly archive v23.0.0-nightly20241016019efe1453 (from today) for source code.

deno compile -A ~/bin/node-v23/bin/npm -o deno-npm
Check file:///home/user/bin/node-v23/bin/npm
error: TS2580 [ERROR]: Cannot find name 'require'.
require('../lib/cli.js')(process)
~~~~~~~
    at file:///home/user/bin/node-v23/bin/npm.ts:2:1

TS2580 [ERROR]: Cannot find name 'process'.
require('../lib/cli.js')(process)
                         ~~~~~~~
    at file:///home/user/bin/node-v23/bin/npm.ts:2:26

Found 2 errors.

We don't have deno bundle anymore in Deno 2+.

Let's bundle with bun first to see if Deno can handle the Ecmascript Module produced

bun build ~/bin/node-v23/bin/npm --outfile=bun-npm-bundle.js

deno compile -A -o deno-npm bun-npm-bundle.js

deno-npm --help
/home/user/bin/node-v23/lib/node_modules/npm/lib/cli/entry.js
error: Uncaught Error: Dynamic require of "/home/user/bin/node-v23/lib/node_modules/npm/lib/cli/entry.js" is not supported
    at file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:35:9
    at file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:547:66
    at module.exports (file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:324:17)
    at module.exports (file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:547:34)
    at file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:552:16
    at file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:19:45
    at file:///tmp/deno-compile-deno-npm/bin/bun-npm-bundle.js:554:16

Update

deno compile -A npm:npm

npm --help
npm <command>

Usage:

npm install        install all the dependencies in your project
npm install <foo>  add the <foo> dependency to your project
npm test           run this project's tests
npm run <foo>      run the script named <foo>
npm <command> -h   quick help on <command>
npm -l             display usage info for all commands
npm help <term>    search for help on <term>
npm help npm       more involved overview

All commands:

    access, adduser, audit, bugs, cache, ci, completion,
    config, dedupe, deprecate, diff, dist-tag, docs, doctor,
    edit, exec, explain, explore, find-dupes, fund, get, help,
    help-search, hook, init, install, install-ci-test,
    install-test, link, ll, login, logout, ls, org, outdated,
    owner, pack, ping, pkg, prefix, profile, prune, publish,
    query, rebuild, repo, restart, root, run-script, sbom,
    search, set, shrinkwrap, star, stars, start, stop, team,
    test, token, uninstall, unpublish, unstar, update, version,
    view, whoami

Specify configs in the ini-formatted file:
    /home/user/.npmrc
or on the command line via: npm <command> --key=value

More configuration info: npm help config
Configuration fields: npm help 7 config

npm@10.9.0 /tmp/deno-compile-npm/bin/node_modules/.deno/npm@10.9.0/node_modules/npm

Bun

What does Bun have to say?

bun build ~/bin/node-v23/bin/npm --compile --outfile=bun-npm

bun-npm --help
npm <command>

Usage:

npm install        install all the dependencies in your project
npm install <foo>  add the <foo> dependency to your project
npm test           run this project's tests
npm run <foo>      run the script named <foo>
npm <command> -h   quick help on <command>
npm -l             display usage info for all commands
npm help <term>    search for help on <term>
npm help npm       more involved overview

All commands:

    access, adduser, audit, bugs, cache, ci, completion,
    config, dedupe, deprecate, diff, dist-tag, docs, doctor,
    edit, exec, explain, explore, find-dupes, fund, get, help,
    help-search, hook, init, install, install-ci-test,
    install-test, link, ll, login, logout, ls, org, outdated,
    owner, pack, ping, pkg, prefix, profile, prune, publish,
    query, rebuild, repo, restart, root, run-script, sbom,
    search, set, shrinkwrap, star, stars, start, stop, team,
    test, token, uninstall, unpublish, unstar, update, version,
    view, whoami

Specify configs in the ini-formatted file:
    /home/user/.npmrc
or on the command line via: npm <command> --key=value

More configuration info: npm help config
Configuration fields: npm help 7 config

npm@10.9.0 /home/user/bin/node-v23/lib/node_modules/npm

Conclusion

node --experimental-sea-config sea-config.json with postject works for CommonJS input. See update. 116.3 MB.

deno compile works. See update. 105.0 MB.

bun build works on first try, out of the box. 96.4 MB.

node and bun compiled executables don't work when the source files are moved from the filesystem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment