This is a serie of reflections on the presentation of Runar Branson Specifically, in the 39:30 he mention side effects as one of the main causes of the lack of composability. Most of the time we need the effects arranged in a different way than the one that forces the composition. For the example that he mentions: choose coffe, pay, serve, we can sustitute the one whose effect we want to reorder (pay) by one non-effectful, (add-to-cart) so that we can make many selections and pay at the end. What we do is to keep in the cart state the items selected.
But there are other reasons why side effects can prevent composition: threading, blocking for wathever reasons: communications, semaphores, callbacks, concurrency, loops waiting for some conditions... These are considered as lost cases for composition and not even mentioned. But transient demonstrates that this is not the case, that it is possible to compose well code that include threads, blockings, callbacks, communications, console interaction etc.
For example I have a program multithreaded, and each part of it could need to interact trough HTTP request and responses. Imagine that the program control a hardware with many device controllers. Can you have the program composable so that a new device controller can be added without changing anything else? To have it done we ned that each one of these controllers send and receive JSON messages and the composition must aggregate them in a single message. codifying a menu with the different devices by defining routes does not count as composition, since it is implied the modification of code in a central element, the routes configuration. The challenge is to have the convenince of the route/menu generated automatically by the composition.
To send a set of JSON registers trough a HTTP response, one option is to simply group them in a list. When the list is finished, i can send the "}". If I use HTTP 1.0 I can close the connection. if it is HTTP 1.1 I can send a empty chunk. Sequentially, the program would wait for some response from the client
sendReceive [reg1,reg2,reg3]And wait for one response
Unfortunately this is not composable. Also, it is not multithreaded so it can not be used to group a set of computations that produce registers. The first is because if I want to compose (monadically, applicative or alternatively) the piece of code than send such list with other piece of code that send some other options, then the program will not work. I have to modify this code and add another fourth register. That is NOT composability.
I can compose them alternatively:
sendReceive reg1 <|> sendReceive reg2 <|> sendReceive reg3The problem is that I do not know what is the last term of these operators. Remember that they are multithreaded. So I can not codify the closing of the registers in a direct way.
(In general each operand could invoque sendReceive repeatedly in different threads. In transient
choose[1..10] >>= sendReceive would send 1 to 10 in different threads)
To allow this, I can store in the state all the threads and his state of execution so that when all the theads are in a blocking statement, for
example, the inner receive of sendReceive, and they have no children, then I'm sure that there is no remaining JSON fragment to send.
This condition is triggered by onWaitThreads, when all the threads are waiting in this branch of the computation:
data InitSendSequence= InitSendSequence deriving Typeable -- to flag, in the state, that JSON content is being sent
sendReceive msg= do
ms <- getRData -- avoid more than one onWaitThread, add "{" at the beguinning of the response
case ms of
Nothing -> do
onWaitThreads $ const $ msend conn $ str "1\r\n}\r\n0\r\n\r\n"
setRState InitSendSequence
msend conn "1\r\n{\r\n"
Just InitSendSequence -> return()
sendChunked msg
r <- receive
delRState InitSendSequence
return rSo onWaitThreads is activated in the first operand and only triggers when the last thread has finished. it send also the opening "{" of the set of registers. When it is triggered it executed all the tasks of the end of the sequence in a chunked encoding.
That composition Also allows to codify arbitrary expressions that uses sendRec within the alternative composition (<|>). That Is the great thing because this allows complete modularity. In this example two different programs interact with the user. One concat strings. The other sum two numbers.
minput is like sendRec, but with a extra parameter. It only receives response when his URL is invoked:
main= keep $ initNode $ concat <|> sum
where
concat= do
h <- minput "one" "first string"
w <- minput "two" "second string"
minput "" (h++w) :: Cloud ()
sum= do
x :: Int <- minput "first" "first number"
y :: Int <- minput "second" "second bumber"
minput "" (show $ x+y) :: Cloud ()For more details see here
The inner receive is also a piece on his own. It is invoked by the message scheduler when the URL of the client request has the proper identifier.
All of this may look contrived to someone. It is very illustrative to study all the "contrived" steps that CPU manufacturers, compiler implementor and Operating system enginers have to do to make possible a simple composition of arithmetical operations in his favorite language, or to insert file IO in the middle of a program.
minput :: Multiple/many sources of input from web, console and autogenerated
A new primitive ´minput` takes input from the console and from web requests. For testing, It will generate arbotrary values using the quickcheck generator.
In this example:
data Test= Test{field1 :: String, field2 :: Int} deriving (Generic, Read, Show, Typeable,Default,ToJSON, FromJSON)
main= keep $ initNode $ do
(str,POSTData test) <- minput "id" "enter a string and a Test value in JSON format"
localIO $ print (str:: String,test :: Test)this present in the console, among other options:
Enter id to: enter a string and a Test value in JSON format "endpt id" for endpoint details
This can be answered by the console or by a web request:
>id hello {"field1": "hello", "field2": 2} <-- written in the console
("hello",Test {field1 = "pepe", field2 = 2} <-- result of the program
> endpt id <-- written in the console
option: endpt
enter the option for which you want to know the interface >"id"
curl -H 'content-type: application/json' -XPOST -d "{\"field1\": \"$field1\",\"field2\": $field2}" http://localhost:8000/id/167624805985580235/0/0/167624805985580235/$stringendpt id return the curl invocation for the endpoint automatically created which get the first parameter from the url and the test value from the post body:
export string="hello"
export field1="pepe"
export field2=2
> endpt id
option: endpt
enter the option for which you want to know the interface >"id"
curl -H 'content-type: application/json' -XPOST -d "{\"field1\": \"$field1\",\"field2\": $field2}" http://localhost:8000/id/167624805985580235/0/0/167624805985580235/$stringwill display in the console:
("hello",Test {field1 = "pepe", field2 = 2})There is more that minput can do. This program:
main= keep $ initNode $ do
h <- minput "one" "first string"
w <- minput "two" "second string"
minput "" (h++w) :: Cloud ()shell prompt >cabal run test5 -- -p start/localhost/8000
Build profile: -w ghc-8.10.4.20210212 -O1
In order, the following will be built (use -v for more details):
- transient-universe-0.7.0.1 (exe:test5) (file tests/test5.hs changed)
Preprocessing executable 'test5' for transient-universe-0.7.0.1..
Building executable 'test5' for transient-universe-0.7.0.1..
[1 of 1] Compiling Main ( tests/test5.hs, /home/user/DAppFlow/dist-newstyle/build/x86_64-linux/ghc-8.10.4.20210212/transient-universe-0.7.0.1/x/test5/build/test5/test5-tmp/Main.o )
Linking /home/user/DAppFlow/dist-newstyle/build/x86_64-linux/ghc-8.10.4.20210212/transient-universe-0.7.0.1/x/test5/build/test5/test5 ...
Enter options to: show all options
Enter ps to: show threads
Enter errs to: show exceptions log
Enter end to: exit
Enter start to: re/start node
Enter cookie to: set the cookie
Enter save to: commit the current execution state
Enter endpt to: info about a endpoint
option: start
hostname of this node. (Must be reachable, default:localhost)? "localhost"
if you want to retry with port+1 when the port is busy, write 'retry':
port to listen? 8000
Connected to port: "8000"
Enter one to: "first string" "endpt one" for endpoint details
> endpt one
option: endpt
enter the option for which you want to know the interface >"one"
curl http://localhost:8000/one/234461250241150446/0/0/234461250241150446/$stringNow in another shell window I use the curl line to enter a value for the $string variable:
shell prompt> curl http://localhost:8000/one/234461250241150446/0/0/234461250241150446/111111
[{ "msg"=""second string"", "req"="{"reqbody":"","requrl":"http://localhost:8000/two/234461250241150446/0/0/234461250241150446/$string","reqheaders":"","reqtype":"GET"}The response has what the second minput produces: a message and the way to invoke it. In this case is a GET request with an URL. if I ivoke it with a value for the $string variable:
shell prompt> curl http://localhost:8000/two/234461250241150446/0/0/234461250241150446/2222
[{ "msg"=""1111112222"", "req"="{"reqbody":"","requrl":"http://localhost:8000//234461250241150446/0/0/234461250241150446","reqheaders":"","reqtype":"GET"}"}]it respond with what the third minput produces: the concatenation of the two strings.
Besides monadic composition, alternative composition allows the composition of entire web programs:
main= keep $ initNode $ concat <|> sum
where
concat= do
h <- minput "one" "first string"
w <- minput "two" "second string"
minput "" (h++w) :: Cloud ()
sum= do
x :: Int <- minput "first" "first number"
y :: Int <- minput "second" "second bumber"
minput "" (show $ x+y) :: Cloud ()running the program like above:
...
Connected to port: "8000"
Enter one to: "first string" "endpt one" for endpoint details
Enter first to: "first number" "endpt first" for endpoint details
> endpt first
option: endpt
enter the option for which you want to know the interface >"first"
curl http://localhost:8000/first/1785466547367704684/0/0/1785466547367704684/$int
> endpt one
option: endpt
enter the option for which you want to know the interface >"one"
curl http://localhost:8000/one/1785466547367704684/0/0/1785466547367704684/$stringYou see that the two URLs for the two programs are available
I have used a simple version of minput which send a String messages in the msg field, but the real version can send anything that may be instance of ToJSON
This program send to the first web request two courses of action, with two different URLs and two different messages:
main= keep $ initNode $ do
minput "init" "init game" :: Cloud ()
left <|> right
where
left= do
minput "left" "go to left" :: Cloud ()
whatever
right= do
minput "right" "go to right" :: Cloud ()
whatever
whatever= localIO $ print "wathever"minput ever print in the console the first options and the way to obtain the URLs of them. getting the URL of the start input:
...
Connected to port: "8000"
Enter init to: "init game" "endpt init" for endpoint details
> endpt init
option: endpt
enter the option for which you want to know the interface >"init"
curl http://localhost:8000/init/5264647062969555436/0/0/5264647062969555436in another console, execute the curl statement, the responses are the two options in the JSON response,with the respective HTTP requests. They have no parameters since they return () (unit).
curl "http://localhost:8000/init/5264647062969555436/0/0/5264647062969555436/()"
[{ "msg"=""go to left"", "req"="{"reqbody":"","requrl":"http://localhost:8000/left/5264647062969555436/0/0/5264647062969555436","reqheaders":"","reqtype":"GET"}"}
,{ "msg"=""go to right"", "req"="{"reqbody":"","requrl":"http://localhost:8000/right/5264647062969555436/0/0/5264647062969555436","reqheaders":"","reqtype":"GET"}"}]