Skip to content

Instantly share code, notes, and snippets.

@dignifiedquire

dignifiedquire/modules.md

Last active August 21, 2016 13:37
Show Gist options
  • Save dignifiedquire/67515fe8d8254b2a041b79bec37a6ac9 to your computer and use it in GitHub Desktop.
Save dignifiedquire/67515fe8d8254b2a041b79bec37a6ac9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
modules.md

Distributing JavaScript

JS IPFS is a large collection of modules that aim to implement IPFS in Node.js and the browser. As such the distributions of these modules has a specific set of constraints.

Our current setup is not bad, and does generate bundles usable in Node.js and the browser, but there are some pain points that need work.

Current Pain Points

  1. Bundles are quite large
  2. Lots of dependencies are duplicated, for example readable-stream is included 12 times in the current js-ipfs browser bundle.
  3. Developers have to know very domain specific configurations to be able to use browserify or webpack.
  4. We break browserify and webpack compat without knowing about it until we get a bug report.

Optimization Goals

  1. Bundle Size
  2. Ease of use for developers embedding the library (i.e. Orbit)
  3. Ease of use for contributors

Module Formats

There are two different module formats for JavaScript modules in main use today.

  1. CommonJS
  • var dep = require('dependency')
  • Only native format in Node.js at the moment.
  1. ES2015 Modules

The current code base uses CommonJS.

Available Tooling

The tooling landscape is quite large today, with things developing and changing quite rapidly. The for us currently relevant tooling is listed below.

Module Bundlers

A module bundler can take in many JavaScript files and generate a bundle, which is usable in the browser.

  1. [Webpack] (CommonJS, ES2015)
  2. jspm (CommonJS, ES2015)
  3. Closure Compiler (CommonJS, ES2015)
  4. Rollup (CommonJS, ES2015)
  5. Browserify (CommonJS)
  6. Babel (CommonJS, ES2015)

ES2015 Transpilers

Transpilers can transform code written with ES2015 features and output code that is usable in ES5 (and lower) environments.

  1. Babel
  2. Typescript
  3. Closure Compiler

A good comparision of the differences in size and runtime can be found in The cost of transpiling ES2015 in 2016.

Proposal

Given the set of constraints mentioned above, the following is a list of steps I suggest to improve and solve our current pain points.

1. Improve build artifacts

Similar to what PouchDB does, the end result for Node.js and the browser should be a single file.

If there are differences between Node.js and browser, modules use two different entry points

  • src/index.js - Original source for node.js
  • src/index-browser.js - Original source the browser

For the builds we target the same places as currently

  • dist/index.js ES5 code for the browser
  • lib/index.js - ES5 code for node.js

but lib/index.js will be a single file, fully transformed rather than still many files such that treeshaking and processing of things like webpack loaders already happend and this is runnable through in node.js directly.

To make tooling aware of what is avaliable, the following should fields should be in package.json

"main": "./lib/index.js",
"jsnext:main": "./src/index.js",
"browser": {
  "./lib/index.js": "./dist/index.js"
},
"jspm": {
  "main": "dist/index.js"
}
Benefits
  • Fully compatabile out of the box, with default configuaration with
    • browserify
    • webpack
    • jspm
    • rollup
Drawbacks
  • Transpiled code in lib/index.js is a bit harder to read as it is now a single large file.

2. Test webpack & browserify in CI

  1. Build with the default configurations for browserify and webpack.
  2. Run the full test suite against these versions.
Benefits
  • We can be sure that our builds are usable by other developers.
Drawbacks
  • CI run time increases.

3. Move to ES2015 Modules

  • Using tools like cjs-to-es6 this is pretty straight forward for our own modules.
  • For dependencies that do not yet publish a build which uses ES2015
  • Enable tree shaking in our webpack build.

Benefits

  • Smaller module size, due to the availability of statically analyzable dependencies and so allowing us to use tree shaking

Drawbacks

  • Not runnable in Node.js directly anymore until they integrate ES2015 modules or you use something like babel-register.

4. Carefully audit the dependency tree

  • Look at all of them
  • Migrate where needed and large enough benefits are clear to ES2105 modules
  • Major culprits that we know about
    • all shims for Node.js functionality in the browser
    • forge
    • web-crypto -> browserify-crypto
    • readable-stream and all users of it
Benefits
  • Only include what we absolutly need
  • Improves tree shaking if we can use dependencies that use ES2015 modules.
Drawbacks
  • Takes time

Resources

Blog Posts

Issues on IPFS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment