async/await is just the do-notation of the Promise monad

promise_monad.md

async/await is just the do-notation of the Promise monad

CertSimple just wrote a blog post arguing ES2017's async/await was the best thing to happen with JavaScript. I wholeheartedly agree.

In short, one of the (few?) good things about JavaScript used to be how well it handled asynchronous requests. This was mostly thanks to its Scheme-inherited implementation of functions and closures. That, though, was also one of its worst faults, because it led to the "callback hell", an seemingly unavoidable pattern that made highly asynchronous JS code almost unreadable. Many solutions attempted to solve that, but most failed. Promises almost did it, but failed too. Finally, async/await is here and, combined with Promises, it solves the problem for good. On this post, I'll explain why that is the case and trace a link between promises, async/await, the do-notation and monads.

First, let's illustrate the 3 styles by implementing a function that returns the balances of all your Ethereum accounts.

Wrong solution: callback hell

function getBalances(callback) {
  web3.eth.accounts(function (err, accounts) {
    if (err) {
      callback(err);
    } else {
      var balances = {};
      var balancesCount = 0;
      accounts.forEach(function(account, i) {
        web3.eth.getbalance(function (err, balance) {
          if (err) {
            callback(err);
          } else {
            balances[account] = balance;
            if (++balancesCount === accounts.length) {
              callback(null, balances);
            }
          }
        });
      });
    }
  });
};

The earliest way to solve this problem was to use callbacks, which caused the dreaded "callback hell", evident from the ugliness of the code above. There are 3 major issues to blame:

1. Explicit error propagation;

2. Keeping track of multiple async values with a counter;

3. Unavoidable nesting.

Almost correct solution: Promises

function getBalances() {
  return web3.eth.accounts()
    .then(accounts => Promise.all(accounts.map(web3.eth.getBalance))
      .then(balances => Ramda.zipObject(accounts, balances)));
}

Promises are first-class terms representing future values. They can be created at will, chained and returned from functions. They almost solve the 3 problems above.

1. Errors are propagated automatically through .then() chains;

2. Promise.all() tracks multiple async values cleanly;

3. Nesting is almost always avoidable by clever usage of .then().

There is one leftover problem, though: .then() still requires nesting in some cases. On my example, for one, the second .then() had to be inside the first, otherwise accounts wouldn't be in scope. This forced me to indent the code right. This situation can sometimes be fixed by passing the value ahead:

function getBalances() {
  return web3.eth.accounts()
    .then(accounts => Promise.all(accounts.map(web3.eth.getBalance).then(balances => [accounts, balances])))
    .then(([accounts, balances]) => Ramda.zipObject(accounts, balances));
}

But that is verbose and, as such, won't work if you have a complex dependency tree, forcing you to indent everything again. In short, Promises solve the callback hell for most cases, but not all.

Correct solution: async/await

async function getBalances() {
  const accounts = await web3.eth.accounts();
  const balances = await Promise.all(accounts.map(web3.eth.getBalance));
  return Ramda.zipObject(balances, accounts);
}

The new async functions allow you to use await, which causes the function to wait for the promise to resolve before continuing its execution. With that, we're able to solve the indenting problem once for all, for any arbitrary code. Now, I'll explain why that is the case. Before, though, let's talk about the actual origin of the problem.

The origins of the callback hell

Many associate the "callback hell" with asynchronous values, but the problem is much more widespread than that: it naturally arises anytime computation must be performed on "wrapped" values that are accessed through callbacks. For a simple example, suppose that you wanted to print all the combinations of numbers from the arrays [1,2,3], [4,5,6], [7,8,9]. This is a way to do it:

[1,2,3].map((x) => {
  [4,5,6].map((y) => {
    [7,8,9].map((z) => { 
      console.log(x,y,z);
    })
  })
});

Notice how the familiar nesting shows up here: it is caused by the same inherent issue that causes the callback hell. Compare [1,2,3].map(x => ...) to promise.then(x => ...). If mapping over nested arrays was as common as dealing with asynchronous values, by 2018 we'd probably have an multi/pick syntax:

multi function () {
  x = pick [1, 2, 3];
  y = pick [4, 5, 6];
  z = pick [7, 8, 9];
  console.log(x, y, z);
}

... which is exactly equivalent to async/await, except that, instead of implementing terms that can be defined in a future, it implements terms that can have multiple values simultaneously.

Monads and do-notation

In some languages such as Haskell and Idris, values which can be wrapped and chained in arbitrary ways are much more common. As such, they had to develop a way to deal with all of them, not just a specific case. Their solution was to specify an "interface" for wrappeable/chainable values, and a special syntax to flatten arbitrary chains of wrapped values.

// Pseudo-code of an interface for wrappeable/chainable values.
// `wrap` receives a value and wraps it.
// `chain` receives a wrapped value, a callback and returns a wrapped value.
interface Wrapper<W> {
  wrap<A>(value:A): W<A>,
  chain<A,B>(value:W<A>, callback: A => W<B>): W<B>
}

The syntax was called the do-notation, and the interface was named "Monad". Thanks those 2 insights, they have, since 1995, been free from their "callback hell" problem. That is why JavaScript finally did it too. If you squint a little bit, JS Promises are just monads of future values that can throw: Promise.resolve(value) implements wrap, and value.then(callback) implements chain. Under that point of view, async/await is just the do-notation for that specific monad, providing the same kind of arbitrary chain flattening. It is, ultimately, the same solution and, given that it worked so well on Haskell, it is reasonable to assume it will work on JavaScript too.

-- Monad is just a fancy name for the Wrapper interface above
class Monad m where
  return :: a -> m a                 -- wrap
  (>>=)  :: m a -> (a -> m b) -> m b -- chain
  
-- Promises are monads, so we can make an instance for it
instance Monad Promise where
  return value       = Promise.resolve value
  value >>= callback = Promise.then value callback

-- Then the do-syntax becomes equivalent to async/await
getBalances :: Promise (Map String String)
getBalances = do
  accounts <- getAccounts
  balances <- getBalance accounts
  return (Map.fromList (zip accounts balances))

Now everything is good and JavaScript is great again, yet I wonder why it took so long. If only the community listened that crazy guy about 5 years ago...

@mdboop damn! You're very right, await is not propagating the errors and I completely missed that. async/await is thus not a full replacement for the do-notation. Everything is terrible again.

It's also nice to note that do-notation can easily be implemented with generators (as co.js and redux-saga do):

function Do(m, f) {
  const gen = f();

  function doRec(v = undefined) {
    const {value, done} = gen.next(v);
    const valueM = value instanceof m ? value : m.of(value);
    return done ? valueM : valueM.then(doRec);
  }

  return doRec();
}

It can be used with any monad that implements .of and .then (being .then == bind), for example:

function generateUserURL(userMaybe) {
  return Do(Maybe, function*() {
    let user = yield userMaybe;
    let name = yield user.name;
    return user.hasPage ? nameToURL(name) : null;
  });
}

or

// Promise.of = Promise.resolve;
Do(Promise, function*() {
  const [user, friends] = yield Promise.all([
    getUserData(userId);
    getUserFriends(userId);
  ]);

  return { ...user, friends };
});

It won't work with the list (Array) monad, though, because JS generators are stateful (Generator.prototype.next mutates the generator). That is, the following example can't be done in JS without special syntax or stateless (rewindable?) generators:

// Array.of = a => [a];
// Array.prototype.then = function(f) {
//    return [].concat.apply([], this.map(f));
// };

Do(Array, function*() {
  let a = yield [1, 2, 3];
  let b = yield ['a', 'b'];
  console.log(a, b);
});

but the following naturally works:

[1, 2, 3].then(a => {
  ['a', 'b'].then(b => console.log(a, b));
});

That is amazingly cool. I'm back to thinking generators are a better solution than async\await because of the lack of error handling on the later.

Edit: but I think it'd be more elegant to just make an object with .of and .then, rather than modifying the prototypes...

@mdboop IMO, async/await becomes a lot more compelling when you introduce logic branching. With Promises or callbacks alone, the logic can become difficult to follow, especially if the flow of logic needs to remerge again.

These challenges are all non-prohibitive, of course, and I personally never thought "callback hell" was as bad as often presented, but I did dislike that switching a call from a synchronous to an asynchronous method somewhere could force an entire region of code to be rewritten in a completely different coding paradigm. Even if the other paradigms had their own elegance to them, it's not ideal for that decision to be driven by the fact that something in the mix happened to be an async call.

With async/await, you at least get the option of continuing to use if/else/try/catch/for/while/etc. if you feel that it makes the code more expressive or natural to write. /2-cents

@mdboop I just recalled having previously sketched about this a little in another gist. See this comment in particular: https://gist.github.com/jmar777/21ace6b8ea9b0cc428fc700faabb77e7#gistcomment-1880603

The TL;DR version of it would probably be this line:

Here's my issue with that, though (and this is where I get into the more abstract side of the argument for async/await): there's no law of nature that states that "if some code includes asynchronous operations, that code must look radically different from code that doesn't".

i disagree with the "lack of error handling" comments. try/catch is the idiomatic way to handle errors -- error callbacks or error codes have always been nothing but workarounds. IMHO the fact that you can now use the try/catch syntax uniformly without having to wonder whether or not your code contains some asynchronous logic is not a problem but a feature.

Cool! I was learning about Haskell's monads today, and had an Eureka moment while taking a bath: "Aha! So >>= is then, async is do, and await is <-. I wonder if someone already made a meme about it in the Internet!". And, so I've typed "js async await then haskell do notation" into google, and found it as the top result. I really liked how your description of how the concepts of Maybe and List monads map to js reality. Thank you all for a great read!

The error is actually propagated, if you have an async function and one of the awaited clauses throws an error, the promise rejects with the error of the first awaited clause. This is the same behavior you would see out of the Either or Maybe monads. I'm not actually sure what error propagation would look like if it wasn't the way it was. The nice thing about this is that it allows you to easily flow between the two syntaxes whenever necessary. I personally think that try-catch was the worst thing that ever happened to programming but fortunately, if I actually need to prevent an error from propagating. I can have the awaited clause have a .catch statement that transforms it back to a successful promise.

I've spent a lot of time in haskell and using fluturejs with their full monadic (lazy) futures api. When I encountered coworkers using async/await, I shied away from it because I didn't like the fact that it obscured the functional semantics of what was going on. Combined with the fact that none of them understood the functional nature of promises made me even more wary of it. But really all that means is that it is a poor teaching tool for the paradigm. I later realized though that it's really just do-notation and at that point I realized that the obscuring the functional semantics didn't mean that you were ignoring them so much as wanting a better syntax for expressing the same concept. Since then, I've been using async/await happily with .then and .catch when those are more convenient.

@CaptJakk I had the exact same thoughts when I first encountered async/await. I thought of Promises as being monads, and so async/await seemed like it was obscuring the underlying bind operation that takes place when one uses .then( ). It's enlightening to realize that async/await is the same type of syntactic sugar as do notation.

Apologies for digging up this old thread, but I've written a babel plugin that some of you may find useful. It leverages the do expression syntax to implement a haskell/scala-esque do/for notation:

https://github.com/pfgray/babel-plugin-monadic-do

Here's a few examples in code sandbox using the wonderful Crocks library:

https://codesandbox.io/s/k143z48q5

That is amazingly cool. I'm back to thinking generators are a better solution than async\await because of the lack of error handling on the later.

Edit: but I think it'd be more elegant to just make an object with .of and .then, rather than modifying the prototypes...

Perhaps not. How about tuples?

// Using the Async class
//
let async = Async()
let sum1 = async.await1Parameter { sum(number1: 1,
                                       number2: 2,
                                       completion: async.closure1Parameter()) }

let (sum2,diff1) = async.await2Parameters { sumAndDifference(number1: sum1 as! Int,
                                                            number2: 3,
                                                            completion: async.closure2Parameters()) }

let (sum3,diff2,mul1) = async.await3Parameters { sumAndDifferenceAndMultiplication(number1: sum2 as! Int,
                                                                                   number2: diff1 as! Int,
                                                                                   completion: async.closure3Parameters()) }

It's a very simple implementation of await/async without promises, simple semaphores are used. Well, as with promises are, too, anyway.
You are not limited by only one single return value and one of them can be error, of course.

Read a short story and there is a small swift.playground, too.

https://medium.com/swift2go/await-in-swift-no-promises-broken-381b531f2843

In practice I've never found error handling with async/await difficult and i would never go back to anything that came before it. Often if it's a recoverable error you can just .catch on the promise you're awaiting and return some dummy value. But try/catch blocks aren't broken just because one has to hoist variable declarations outside of them. Not gonna lie though, a scopeless try/catch would be nice.

Don't we need any special treatment for balances <- getBalance accounts?
I assume getBalance :: Account -> Promise Balance so we probably want something like
balances <- mapM getBalance accounts to get a Promise [Balance]?

It won't work with the list (Array) monad, though, because JS generators are stateful (Generator.prototype.next mutates the generator) [...]

It is possible to make it work for Array, although it is probably slow:

function Do(m) {
  return function(gen) {
    return (function run(as) {
      return function(a) {
        const g = gen();
        as.forEach(a => g.next(a));
        const { value, done } = g.next(a);
        if (done) {
          return m.of(value);
        }
        return value.flatMap(run(as.concat(a)));
      };
    }([])());
  };
}

Array.do = Do(Array);

const result = Array.do(function* () {
  const x = yield [1, 2];
  const y = yield [3, 4];
  return x + y;
});

console.log(result); //=> [4, 5, 5, 6]

Everything is just CPS

Perfect!!