The final result: require() any module on npm in your browser console with browserify

This article is written to explain how the above gif works in the chrome (and other) browser consoles. A quick disclaimer: this whole thing is a huge hack, it shouldn't be used for anything seriously, and there are probably much better ways of accomplishing the same.

Update: There are much better ways of accomplishing the same, and the script has been updated to use a much simpler method pulling directly from browserify-cdn. See this thread for details: mathisonian/requirify#5

inspiration

My inspiration for building this was Max Ogden's Requirebin, which allows users to use a browser based editor to run custom javascript in the browser (including javascript that had require() statements that would normally need to be pre-processed using browserify).

Requirebin is a great site, but I found myself wondering if it was possible to do something like this with an interactive REPL.

digging in

After browsing the source code for requirebin, it became apparent that the key to making it work was a project called browserify-cdn, an HTTP-based service which given a URL, will return in the response body a string of javascript representing a browserified bundle of an npm library.

That is, given a URL like

/standalone/lodash@latest

browserify-cdn will bundle up lodash and all of its dependencies in the same way that browserify would if we were doing require('lodash') in a client side project. The browserify-cdn folks are even nice enough to host a version of this at http://wzrd.in/, so if you visit http://wzrd.in/standalone/lodash@latest you can see the exact output of this project.

requiring() against browserify-cdn

At some point we are going to have to define a request() function for the browser. Let's skip over the details of this for right now, and assume it looks something like

var request = require('superagent');
var require = function (moduleName) {
    request("http://wzrd.in/standalone/" + moduleName + "@latest", function(error, res, body) {
        if(error) {
            return console.log(error);
        }
        eval(body);
    });
};

this includes the client-side request library superagent for brevity, but the gist of it is that this fetches the browserified bundle for whatever library we just requested, and evaluates it.

the problem with scope

Unfortunately, browserify bundles are very nice and friendly, and everything is kept in a reasonable scope. Our code in the browser console doesn't really have access to anything happening inside the browserify bundle. So how can we get around this?

One easy way would be to take things from inside the browserify bundle scope and put them outside, like onto the global window scope.

[remember that disclaimer?]

But don't we need access to the inside of a browserify bundle at some point? It turns out, not necessarily, especially given the fact that anyone has the ability to publish any sort of contrived npm module that they can dream of.

It would be really great if for any module that we wanted to require() in the browser, there was another module that all it did was require our desired library and attach it to the window object.

Something like

window = window || {};
window.lodash = require('lodash');

or more generally (using underscore templating syntax)

window = window || {};
window.<%= moduleName %> = require('<%= moduleName %>');

but we don't always want the names to be the same (it's often the case that we want something like var _ = require('lodash')), so we can add one more variable

window = window || {};
window.<%= name %> = require('<%= moduleName %>');

autopublishing modules

It seemed possible to programmatically publish modules like that, so I built a library to do it. Requirify is a simple npm module that can be used like this

var requirify = require('requireify');

requirify('_', 'lodash', function(err, moduleName) {
    // moduleName is requirify-_-lodash, this
    // module is now published on npm.
})

and it publishes modules of the form described above to npm for you. I set up a little bot to handle all of these, and made sure to namespace them with requirify- in front so that they aren't polluting names that other people would want to use.

So, for example https://www.npmjs.org/package/requirify-_-underscore is just a 2-line npm module:

window = window || {};
window._ = require('underscore');

And since these modules exist on npm, they also exist on browserify-cdn!

a proxy server

The remaining thing to do is to make sure that the publishing step happens before we make a request to browserify-cdn. One solution for this is to create a proxy server that runs requirify on the module before proxying the request to browserify-cdn. This is fairly straigtforward and the entire implementation can be seen here:

/*!
 * Module dependencies.
 */
var requirify = require('requirify');
var request = require('request');


exports.index = function (req, res) {

    var moduleName = req.params.modulename;
    var varName = req.params.varname;

    requirify(varName, moduleName, function(err, requrifyModuleName) {
        if(err) {
            return console.log(err);
        }

        var newurl = 'http://wzrd.in/standalone/' + requrifyModuleName + '@latest';
        request(newurl).pipe(res);

    });
};

a typical express route, taken from https://github.com/mathisonian/requirify-web/blob/master/app/controllers/home.js. The server is up and running on heroku currently. It checks if the requirify-X-Y module already exists, and if not it will publish a new one before continuing on to proxy your request.

injecting the require() function

And finally, we have to inject the require() function into the browser. There is a script hosted on amazon s3 that properly defines the function: https://s3.amazonaws.com/s3.mathisonian.com/javascripts/requirify-browser.js.

This script can be injected into the body of web pages using a chrome extension (https://chrome.google.com/webstore/detail/requirify/gajpkncnknlljkhblhllcnnfjpbcmebm) or a simple javascript boomarklet.

The final require() function looks like

var request = require('superagent');
var require = function(name, moduleName) {
    moduleName || (moduleName=name);
    console.log("Fetching " + moduleName + "... just one second");
    request("https://evening-chamber-1845.herokuapp.com/" + name + "/" + moduleName, function(er, res, body) {
        if(error) {
            return console.log(error);
        }
        var r = eval(body);
        console.log("Finished getting " + moduleName);
    });
};

so you have the ability to require('_', 'lodash'). This will fetch lodash from npm and assign it to the _ variable.

update

The script has been updated to interact directly with browserify-cdn, and now looks like this:

var request = require('browser-request');

window.require = function(name, moduleName) {
    _require = require;

    if(!moduleName) {
        moduleName = name;
    }

    console.log('Fetching ' + moduleName + '... just one second');
    request('http://wzrd.in/bundle/' + moduleName + '@latest/', function(er, res, body) {

        require = null;
        eval(body);
        window[name] = require(moduleName);
        require = _require;
        console.log('Finished getting ' + moduleName);
    });
};

kiruto/index.md