Asynchronous exceptions and de-inverted management of resources in transient

resources.md

Asynchronous exceptions and management of resources in transient

NOTE: All this has been superseeded by finalizations

I was looking at the last article of Michael Snoyman about asynchronous exceptions. Proper handling of resources in long term programs such are servers demands very accurate management of resources. In transient where many threads are spawned and sometimes killed asynchronously, this is even more important.

So I first tried to create a version of bracket for the transient monad:

bracket
        :: IO a              -- ^ computation to run first ("acquire resource")
        -> (a -> IO b)       -- ^ computation to run last ("release resource")
        -> (a -> TransIO c)  -- ^ computation to run in-between
        -> TransIO c         -- returns the value from the in-between computation
bracket before after thing = ???? do this

Then I realized that for synchronous "normal" exceptions acquisition of release in transient was solved already in this piece of code.

There, withResource is very similar to bracket; In terms of usage, it performs an initialization, then executes the computation and the finalizes the resource: This can be written as:

withResource before after thing= do
      res <- before
      thing res
      after res

Note that withResource does not return the result of thing res, since it is intended to use the WHOLE rest of the computation (the continuation). For this purpose, it uses react. Using our redefined withResource, if we write:

do
    res <- react (withResource before after)  (return ())
    blah res
    blah blah
    etc

The thing part becomes the continuation, that is: (\res -> do blah res; blah blah; etc)

So the whole continuation, which runs the Transient monad, is executed before the releasing of resources.

Since the handling of threads in transient catch exceptions and continue doing any cleanups necessary, it also ever executes the after, no matter if the finalization was normal or by a synchronous or asynchronous exception. So far so good.

Solved?

No. Two bad news: first an asynchronous exception can interrupt the allocation of resources in before or, what is worst, between the complete execution of before and before the init of the continuation. Since the continuation has not started, the thread has not set the handling of exceptions that the continuation has... Sooo the cleanup of after is not executed.

What is the solution? bracket itself!

Instead of the hand-coded withResource I can use bracket, which has almost the same signature:

do
    res <- react (bracket before after)  (return ())
    blah res
    blah blah
    etc

I mean, bracket and my withResources structures are equal except the return value, which is discarded since thing is the whole computation. bracket also does what is needed: it prevents exceptions between the allocation and the set-up of interruption handlers of the continuation (done by the library)

And here comes the second problem: I want to deallocate resources as early as possible, as soon as is not needed. This schema does not allow it since the deallocation of after is the last action that the thread performs, by the definition of react.

So let's run the computation isolated in his own thread. After all, that is what the article recommends. For this purpose I use a transient primitive called collect:

do
    v <- useResources $ do
            res <- react (bracket before after)  (return ())
            blah res
    blah blah v
    etc 

    where
    useResources proc= head <$> collect 1 proc

Now the release of resources happens as soon as useResources finalize, so we have limited the lifetime of the resource. We now can free it as soon as we could.

-How? what happens?

collect spawn his second argument as a separated thread which dies and executes last, and wait for the return values returned by the spawned computation, one value in this case: the return value of blah res. Since collect return a list -of one in this case- in a monadic value TransIO [a], I get the element, the head of it.

If there is an exception, the program would not continue, but, since the thread finish, last is executed to free the resource. Optionally, it can execute further exception handlers upwards if there is any that match. See the "exceptional" mechanism for managing exceptions in Transient.

So to summarize, we have two primitives useResources -which is a particular usage of collect- and tbracket, which is the combination of react and bracket. It is a bracket de-inverted by react:

-- | De-unverted bracket wich expose the resource to the continuation and free the resources at the en of his thread
tbracket :: :: IO a     -- ^ computation to run first ("acquire resource")
        -> (a -> IO b)  -- ^ computation to run last ("release resource") 
        -> TransIO a
tbracket before after= react (bracket before after)  (return ())

-- | run the computation on a new thread and wait for the result
useResources :: TransIO a -> TransIO a
useResources proc= head <$> collect 1 proc

So what?

Is tbracket better than bracket at the end of the day?. Well, it allows for more transparent use of resources and more composability. That means that it is less invasive, it does not change your way to do things because the functionality of managing resources is introduced:

openClose name mode= tbracket (openFile name mode) hClose


do
     handle <- openClose "config.csv" ReadOnly
     dosomething handle
     etc

openClose open the file and close it at the end of the thread. It seems to me much more natural than:

-- As is defined in System.IO
withFile name mode = bracket (openFile name mode) hClose

do
    r <-withFile "config.csv" ReadOnly $ \handle -> do
                         dosomething handle
                         etc
                         ....

In the second case, there is a break in the monadic composition by the fact that a primitive bracket should call your code as a callback when the environment has been set for interruptions. When such abrupt changes are pervasive for whatever functionality, the code becomes confusing.

I can convert a program which uses open and substitute it for openClose with a few keystrokes since the structure does not change. I added a new functionality/effect without reshaping the code. But I should change the structure to use withFile.

There are other advantages of enhanced composability inherent to the transient monad such is that exceptions propagate across threads, seamless parallelism and concurrency etc. But an enhanced IO monad could have it as well. The inherent advantage is the de-inversion operated by the use of continuations, which restores the "normal" way of doing things. Inverted primitives such is bracket leave to the programmer the work of managing the continuation by hand.