Meteor Async Guide

meteor-async.md

From Meteor's documentation:

In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node. We find the linear execution model a better fit for the typical server code in a Meteor application.

This guide serves as a mini-tour of tools, trix and patterns that can be used to run async code in Meteor.

Basic async

Sometimes we need to run async code in Meteor.methods. For this we create a Future to block until the async code has finished.

To use Future starting from Meteor 0.5.1, you have to run the following code in your Meteor.startup method:

Meteor.startup(function () {
  var require = __meteor_bootstrap__.require
  Future = require('fibers/future');

  // use Future here
}

And starting from 0.6.5, you have to run the following:

Meteor.startup(function () {
  Future = Npm.require('fibers/future');

  // use Future here
}

This pattern can be seen all over Meteor's own codebase (pre 0.6):

Meteor.methods({
  asyncJob: function(message) {
  
    // Set up a future
    var fut = new Future();

    // This should work for any async method
    setTimeout(function() {

      // Return the results
      fut.ret(message + " (delayed for 3 seconds)");

    }, 3 * 1000);

    // Wait for async to finish before returning
    // the result
    return fut.wait();
  }
});

This pattern can be seen all over Meteor's own codebase (POST 0.6):

Meteor.methods({
  asyncJob: function(message) {
  
    // Set up a future
    var fut = new Future();

    // This should work for any async method
    setTimeout(function() {

      // Return the results
      fut['return'](message + " (delayed for 3 seconds)");

    }, 3 * 1000);

    // Wait for async to finish before returning
    // the result
    return fut.wait();
  }
});

Code

Parallel async

Sometimes we want to run more than one async methods in parallel and wait for them all to complete before returning:

Meteor.methods({
  parallelAsyncJob: function(message) {

    // We're going to make http get calls to each url
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    // Keep track of each job in an array
    var futures = _.map(urls, function(url) {

      // Set up a future for the current job
      var future = new Future();

      // A callback so the job can signal completion
      var onComplete = future.resolver();

      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Do whatever you need with the results here!
    
        // Inform the future that we're done with it
        onComplete(error, result);
      });

      // Return the future
      return future;
    });

    Future.wait(futures);
  }
});

Code

If you want to collect results from parallel async jobs you'll have to do a little more work:

Meteor.methods({
  parallelAsyncJob: function(message) {
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    var futures = _.map(urls, function(url) {
      var future = new Future();
      var onComplete = future.resolver();
  
      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Get the title if there was no error
        var title = (!error) && getTitle(result);
    
        onComplete(error, title);
      });
  
      return future;
    });

    // wait for all futures to finish
    Future.wait(futures);

    // and grab the results out.
    return _.invoke(futures, 'get'); 
  }
});

Code

Unblocking Meteor.methods

Each approach described so far breaks from Meteor's default synchronous programming style, but requests from a given client are still processed one at a time. If you have a method that could take a long time call this.unblock() and subsequent requests will be run in a new fiber.

Meteor.methods({
  moveForkliftToBuilding9: function(message) {
    this.unblock();
    forklift.moveTo({ building: 9 })
  }
});

Note: borrowed this method name from Matt Debergalis because I think it's hilarious

Accessing Meteor's full environment from async callbacks

If you need to access Meteor's full environment in an asynchronous method's callback Meteor provides a wrapper called Meteor.bindEnvironment:

TODO make better example

var fn = function(error, result) {
  console.log('done ' + result);
};

fn = Meteor.bindEnvironment(callback, function(e) {
  var message = "Something went wrong! " +
                "Everything is broken! " + 
                "Your life is ruined!";

  console.log(message, e.stack);
});

myAsyncMethodThatNeedAccessToMeteorEnv(fn);

Packaging async node modules

TODO

Abstractions

At this point I'm leaving this purposely unfinished because until I have opportunities to use these techniques more than once, in real apps, I don't want to guess what the right abstractions are. Hopefully people will write awesome smart packages for this stuff and we can find out together what will be most effective and then campaign for our favorites to be be included in core.

TODO

Include all useful info mentioned by Matt here

Instead of example like asyncJob and parallelAsyncJob come up with meaningful names and implementations

Contributions

Thanks to the following people for their help with this:

Tom Coleman @tmeasday

Resources

http://stackoverflow.com/a/11510874