A description of what I am trying to plan to do with Mono Coroutines.

README.md

I have written a wrapper for libuv in C#. libuv is the IO abstraction for node.js.

libuv basically exposes an event loop for IO (it abstracts all the different platforms away).

Now the problem that I face is that I have a callback based API everywhere and that sucks, because of the constant indentation and rather hard programming against the API:

var server = new TcpListener();
server.Bind("127.0.0.1", 8000);
server.Connection += () => {
	var tcp = server.Accept();
	tcp.Data += (data) => {
		Console.Write(Encoding.ASCII.GetString(data.Array, data.Offset, data.Count));
		server.Close();
		tcp.Close();
	};
	tcp.Resume();
};
server.Listen();
var client = new Tcp();

client.Connect("127.0.0.1", 8000, (ex) => {
	client.Write("Hello World\n", (success) => {
		client.Close();
	});
});

Loop.Default.Run();

Now that sucks. Callbacks everywhere...

There are helper functions, which queue operations, so you can get a shorter version:

var server = new TcpListener();
server.Bind("127.0.0.1", 8000);
server.Connection += () => {
	var tcp = server.Accept();
	tcp.Data += (data) => {
		Console.Write(Encoding.ASCII.GetString(data.Array, data.Offset, data.Count));
		server.Close();
		tcp.Close();
	};
	tcp.Resume();
};
server.Listen();
var client = new Tcp();

client.Connect("127.0.0.1", 8000); // this just queues it and falls through
client.Write("Hello World\n"); // same here
client.Shutdown(); // same here

Loop.Default.Run(); // the only real blocking operation

The downside is that you cant queue a read, if you want to have access to the read data immediately after a read.

But C# has async/await, right? So I tried to wrap and got:

using System;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using LibuvSharp;
using LibuvSharp.Threading.Tasks;

namespace Test
{
	class MainClass
	{
		static IPEndPoint ep = new IPEndPoint(IPAddress.Any, 8080);

		public static async Task Server()
		{
			try {
				var server = new TcpListener();
				server.Bind(ep);
				server.Listen();

				var client = await server.AcceptAsync();

				client.Write(Encoding.ASCII, "Hello World!");
				var str = await client.ReadStringAsync();
				Console.WriteLine("From Client: {0}", str);

				client.Shutdown();
				server.Close();
			} catch (Exception e) {
				Console.WriteLine("Server Exception:");
				Console.WriteLine(e);
			}
		}

		public static async Task Client()
		{
			try {
				var client = new Tcp();
				await client.ConnectAsync(ep);

				client.Write(Encoding.ASCII, "Labas Pasauli!");
				var str = await client.ReadStringAsync();
				Console.WriteLine("From Server: {0}", str);

				client.Shutdown();
			} catch (Exception e) {
				Console.WriteLine("Client Exception:");
				Console.WriteLine(e);
			}
		}

		public static void Main(string[] args)
		{
			Loop.Default.Run(async () => {
				Console.WriteLine("Starting example.");
				await Task.WhenAll(Server(), Client());
				Console.WriteLine("All finished.");
			});
		}
	}
}

Now the API looks very nice. Everything what happens here happens in the same Thread.

Code between two awaits is atomic (as in it won't get interrupted by other functionality). This gives the programmer a very good idea on how and when to make changes to state (can be even global).

It would be awesome except there are few problems I face:

• I am basically going against the established async/await API which usually executes something in another thread and posts the result back in this one (I am in an event loop, I have to do everything in the same one, at least this is my goal).
• All the async/await existing API within the .NET Framework won't work with this (or at least I don't know how to make it work).
• A lot of code has been written with the blocking standard API and together ... and that won't, becaues everything what blocks sucks.

But I have discovered Mono Coroutines. Well, I have known about them for a very very long time, but thinking about how to tackle the problems described above I was enlightened last week. I haven't gotten a working example yet, because I basically need to rewrite libuv in C#, since I need to have absolute control of the stack (after all I am fidling with the stack using Monos Coroutines), but here is what my desired API will look like:

var server = new TcpListener();
server.Bind("127.0.0.1", 8000);
server.Connection += () => {
	var tcp = server.Accept();
    byte[] data = new byte[512];
    // this is the most interesting part
    // we stop executing with the coroutine API here and 'jump' back into Loop.Default.Run
    // where we process process other events, until this read event is fetched, then we use
    // a coroutine to execute the code right after server.Read
    int read = server.Read(data, 0, data.Length);
	Console.Write(Encoding.ASCII.GetString(data, 0, read);
    tcp.Close();
};
server.Listen();
var client = new Tcp();

client.Connect("127.0.0.1", 8000); // this is absolutely blocking, it executes the loop until the connect is finished
client.Write("Hello World\n"); // this as well
client.Close();

Loop.Default.Run();

Everything happens in one and the same thread.

It is a bit harder to reason about code, because only operations between two "blocking" calls are atomic, so in the Connection event everything before the read happens in one pass and after the read as well (other events might occure between those two steps).

The upsides:

• I can use a SynchronizationContext for await/async which works with all the default .NET libraries.
• I can use libraries which were written with the blocking API, since I perfectly emulate all blocking operations.
• It should be faster than async/await (less GC pressure, generated async code is usually really fat, creates objects on its own, needs constant creation of Task objects).

A note on the performance:

I tried to benchmark the continuation approach by comparing it to calling a delegate. The coroutine test had a delegate, but I stored and restored in the middle of it, I'll edit this post later to add the test.

Calling the naked delegate was around 2 times faster, we are talking about 1,6 compared to 3.5 million iterations on a single CPU Core on my old machine.

Another positive thing of coroutines opposed to async/await is that if an application using linux epoll tries to read most of the time it is a non blocking operation (if the socket buffer has something in it) so we do not even need to do a coroutine store/restore, just make a read call which does not bloc. Using async/await API we still would need to create a Task and return it immediately set, which would create pressure on the GC, since we create objects for every event, and we have a lot of events in a system which is used.