callback-vs-promise-errors-story.md

Second example at

https://www.joyent.com/developers/node/design/errors

in the section "(Not) handling programmer errors"

A database (or other) connection may be leaked, reducing the number of future requests you can handle in parallel. This can get so bad that you're left with just a few connections, and you end up handling requests in series instead of concurrently.

Ok, so I have code that opens a connection, runs a query, then processes the results.

The callback story:

function doSomething(processingFunction, cb) {
  db.connect(opts,(err, conn) => {
    if (err) return cb(err);
    conn.query(someQuery, (err, res) => {
      if (err) return cb(err);
      var processed = processingFunction(res);
      conn.close(err => cb(err, res))
    });
  })
}

First programmer error

If the conn.query call fails we invoke the continuation callback without cleaning up the connection. Nothing threw to warn us about this programmer error, so clearly its not a programmer error? Unfortunately, the connection file handle is leaked.

Second "programmer" error

processingFunction may throw. That may be a programmer error within processingFunction like a typo in the function code. Or it may simply be the fact that processingFunction didn't like our data for some reason and threw an error. Maybe the database rows didn't have the right data for that kind of processing. Its common for node functions to synchronously throw if they don't like their input data.

Luckily for us, the process will crash, so the handle will not leak. If we're not so lucky and, for example, this process serves a webpage, and for some reason the user starts refreshing the page like mad, they might kill all our workers immediately, crippling the website.

Additionally, if this process is also serving thousands of other people, their work may be lost or corrupted. This is unacceptably bad

Third programmer error

We simply call the continuation if there was an error closing the connection. What if the connection wasn't closed? Then we're leaking a handle. Clearly this should throw and crash the process somehow. But since its an async operation, nothing throws.

Now lets look at the promise story. I will cheat a little and use bluebird's disposer/using implementation. Its fairly simple to implement from scratch (unless you want an implementation that can deal with allocating multiple resources in parallel):

The promise story

// library code
function dbConnect(options) {
  return Promise.resolve().then(_ => db.connect(options)).disposer(conn => conn.close())
}

// user code
function doSomething(processingFunction) {
  return Promise.using(dbConnect(options), conn => {
    return conn.query(someQuery).then(processingFunction)
  })
})

Now, assuming the library code is correct (which by the way is much easier to ensure, because its just one piece of code that provides the database connection as opposed to hundreds that will use it):

First programmer error

Impossible to happen. After the using block completes, the connection will close, no matter what.

Second "programmer" error

Can still happen. Connection gets closed, no matter what. The process doesn't crash either. We can happily continue.

Third programmer error

Its possible that closing the connection fails. In this case, the bluebird disposer will actually crash the process. Because there really is no other way to dispose of a file handle that refuses to close.

The burning question

So the burning question is then, how do we deal with the obvious programmer errors that are sources of exceptions?

• tried to read property of "undefined"
• called an asynchronous function without a callback
• passed a "string" where an object was expected
• passed an object where an IP address string was expected

Answer: with a typechecker such as Flowtype, TypeScript, Closure (google's compiler) etc. Alternatively, excellent test coverage.

It may seem like this is not the sensible answer - after all, we are using a dynamic language, we don't want those type checkers. But really, all of the above errors are type errors. If we truly have a problem with the possibility of those errors happening, then a typechecker really is the best tool to deal with them. Even better, it would catch them before they ever run.

The above typecheckers offer varying levels of flexibility and escape hatches. You can still write dynamic code, if you really want or need to do that in some places.

Its of course always good to have tests (even when using a type checker) for the things that a checker isn't able to detect.

And if our code is generally written to be resilient, not brittle, especially when dealing with resources and shared state, the remaining programmer errors that we failed to catch would cause much less damage overall. If processingFunction really did have that typo, we can still provide some guarantees: our connection will still get closed.

The real problem

todo: discuss the problem of shared state