My thoughts on making the Web more composable like UNIX

composable_web.md

The Composable Web Proposal

Serverless infrastructure like AWS Lambda and Google Cloud Functions have made it much cheaper for developers to offer server-side code for public consumption without keeping a server always running.

If these functions could be declared as stateless or deterministic, costs can be brought down even more because only the first invocation needs to be executed. Cached response could be returned for future invocations with the same input arguments.

All modern browsers support URL lengths of thousands of characters, even on mobile. A lot of data can be embedded and passed around directly in the URLs (instead of passing identifiers which requires a look-up which costs server time).

So here's a thought:

• Write your cloud functions/lambdas to handle public HTTP requests. The arguments should be taken only from the URI path and query string

• Configure your infrastructure to return cached responses for functions. If your vendor doesn't support it natively, invoke the cache from within your function itself. This can be made easier if a resourceful org steps up to run a shared cache for us all. Eventually, competition among infra providers will make this a standard feature of serverless products.

Example: Meme generator

Let's say you want to make a meme-generator. We can do this by composing these 3 functions:

• f1: Extract an image from a YouTube video at a specific timestamp
• f2: Crop an image given its URL
• f3: Add some text to an image given its URL

This is how it could work:

1. https://lambda.aws.com/video-to-image?format=jpg&youtube=https://you.tube/234345234?ts=231
2. https://functions.cloud.google/image-cropper?url={1}&width=500&height=300
3. https://func.azure.cloud/meme-text?image={2}&top=Well&bottom=That+escalated+quickly

If we have caching enabled for the third function, all 3 function invocations only need to happen once for this specific meme. So it reduces down to serving a static file (which is trivially cheap now and even free thanks to Netlify & GitHub). Having caching enabled for f1 and f2, will save costs when only f2 or f3 invocation needs to be customized.

Care needs to be taken for safely encoding URL paths and query strings to be passed as paths and query strings. If the encoding of arguments could be standardized even further, we can even build GUI apps like Yahoo Pipes.

More examples and ideas

• Data format converters: JSON, CSV, Excel etc
• Charts or infographic generation using public data
• Data extraction and processing from audio/video using Google' ML Kit
• Functions which generate code and/or evaluate safe/sandboxed code. This can avoid the need for every developer to set up lambda functions. You could write the code in a Gist and pass its SHA (thus making it deterministic and cacheable).

Extra

• We should allow the users to be able to understand how it all works. Passing a standard parameter like show-source in any of the function calls should return both a machine-readable response with a description and link to the code repository, and a human-readable response which escalates the show-source parameter up the invocation chain and presents a page that explains the entire pipeline. This needs more thought but it can help make the Web more comprehensible and attracts more contributors to enhance your lambda functoins.

What do you think?

Tell me in the comments.

Have you looked into how CloudFlare Workers do things?