Skip to content

Instantly share code, notes, and snippets.

@crabmusket
Last active February 15, 2025 04:19
Show Gist options
  • Save crabmusket/6a63eec5aa9d92649f66614f09777c6a to your computer and use it in GitHub Desktop.
Save crabmusket/6a63eec5aa9d92649f66614f09777c6a to your computer and use it in GitHub Desktop.

JSON-RPC pipeline batches

Typical RPC implementation

Say we want to implement an RPC service for basic maths operations. For example, let's calculate the value of ln(e^2). This calculation has several steps in our maths API:

  1. Get the value of e
  2. Square e
  3. Calculate the ln of the result

In a typical RPC-style server, each of those steps would be an HTTP request:

const math = getRPCClient();
const e = await math.e();
const e_squared = await math.pow({base: e, exp: 2});
const ln_e_squared = await math.log({arg: e_squared, base: e});

Introducing pipelining

In order to cut down on HTTP round trips, we'd like to be able to send a whole expression to the server rather than one expression at a time.

JSON-RPC allows us to send batch requests to fit many calls into a single HTTP request, but that doesn't help us here because our calculations need to use the results of previous requests. 'Pipelining' would let us specify how to use results as inputs to requests in the same batch.

Example JSON-RPC request:

[
  {"jsonrpc": "2.0", "method": "math.e", "id": "e_val"},
  {"jsonrpc": "2.0", "method": "math.pow", "params": {"base": "$e_val", "exp": 2}, "id": "e_squared"},
  {"jsonrpc": "2.0", "method": "math.log", "params": {"arg": "$e_squared", "base": "$e_val"}, "id": "ln_e_squared"}
]

and response:

[
  {"jsonrpc": "2.0", "result": 2.718281828459, "id": "e_val"},
  {"jsonrpc": "2.0", "result": 7.389056098930, "id": "e_squared"},
  {"jsonrpc": "2.0", "result": 2, "id": "ln_e_squared"}
]

Note the:

  1. Use of request results as params by prefixing their IDs with $
  2. Use of "$e_val" where a number was expected in math.pow
  3. Reuse of e_val in calls to both math.pow and math.log

The client should be able to send requests in any order inside the batch, and the server will resolve the dependency ordering between their variables.

Client

In order to take maximum advantage of pipelining, it'd be nice to abstract the process of creating request IDs and constructing the batch request. A DSL based on ES6 Proxies or generated code could provide a very readable interface.

This would be nice:

const math = getRPCClient();
const e = math.e(); // e is now some kind of Promise
const e_squared = math.pow({base: e, exp: 2});
const ln_e_squared = math.ln({arg: e_squared, base: e});

// the client doesn't actually send the request until you call get()
ln_e_squared.get().then(ln_e_squared => console.log(ln_e_squared === 2));
// not sure if the name shadowing is good style or not

// the intermediate values are Promises too, so this works:
Promise.all([e, e_squared]).then(...)

// but is there a more elegant way to get intermediate results? how about:
ln_e_squared.get({e, e_squared}).then((result, {e, e_squared}) => ...);

// or similarly:
ln_e_squared.get().then((result, results) => console.log(results.get(e)));
// results.get looks up the uid of e, generated when e was created

// or what if we automatically filled in the results after get()?
ln_e_squared.get().then(() => console.log(e.result));

// I don't like this heaps because it relies on the values being mutable
// on the other hand it's sort of just like memoisation, if functions are pure

// here's an example where that may go wrong:
const rand = math.random();
const rand_0_to_10 = math.mul([rand, 10])
rand_0_to_10.get().then(() => console.log(rand.value));
rand_0_to_10.get().then(() => console.log(rand.value));
// are the two rand.values a race condition?
// actually probs not thanks to JS being single-threaded
// BUT if we use rand.value outside those callbacks, then it's not well known
@tooolbox
Copy link

tooolbox commented May 7, 2022

I'm a fan of jsonrpc, but I recently found myself looking for pipelining since it exists in cap'n'proto. Do you know if anyone's done anything like what you laid out?

@southpolesteve
Copy link

Found this gist after building a very similar idea! Here is the POC on cloudflare workers https://jsrpc-browser.southpolesteve.com/. Code linked. It needs much more built out but I think the core idea is solid.

Captnproto is what inspired me too. The creator works with me at Cloudflare.

@crabmusket
Copy link
Author

crabmusket commented Feb 15, 2025

@southpolesteve Wow, I'm envious you get to work with Kenton! His career and projects have been really interesting. I'm currently actually getting interested in Sandstorm development as I have a need for self hosting. It's so cool how you can see the precursor to Durable Objects there.

I'll check out your repo, as I'm likely to start looking back into this sometime soon. However, I am less interested in RPC pipelining these days than sync engines.

@tooolbox check out CloudFlare's JSRPC which they're building in their workers platform. It's this but not built in JSON-RPC.

JSON-RPC is a nice spec but it doesn't give you a lot. Basically just id. The rest of the owl is yours to draw. More here https://crabmusket.net/2024/lessons-from-the-json-rpc-mailing-list/

@tooolbox
Copy link

Looks very cool, but is JSRPC usable only with Cloudflare? I was hoping for something a little more BYOBackend. I mean, I suppose original capnproto fits that description, but no in-browser support. So there may be a little square peg round hole situation for me.

@crabmusket
Copy link
Author

I suspect it's part of workerd which is open source but not off-the-shelf usable afaict. I mentioned it more as inspiration!

@southpolesteve
Copy link

Sorry the naming is really confusing. The link I posted has no dependency on Cloudflare. Its only deployed there to show it off. Open it in the browser and check out the web sockets messages. Its doing something very similar as described in this gist. You could rework it to be served by express or even a non-JS backend.

Cloudflare also has a thing called JSRPC that makes it easy to talk between two workers. That system is implemented with capnproto under the hood and is part of workerd which is open source.

Part of why I was experimenting is it would be cool to extend JSRPC out to the browser. But implementing full captnproto to the browser is a tall order. What I made is a simpler attempt at solving that problem on top of JSON-RPC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment