lambda connection problem

lambda.md

AWS Serverless Network Connection Management

Problem Statement:

Serverless compute targets (e.g. AWS lambda) that initiate network connections have no built-in way of offloading that connection management in the case of long polling, websockets, http2, etc.

This means that a reactive lambda that makes an outbound network connection spends most of its execution time in an endless IO wait.

AMZN APIGW can take incoming websocket connections and translate those requests into a request/response model. This is desirable! What would be great is allowing that for outbound connections as well.

Currently the only way to do this is with an out of band proxy (typically nginx).

I would pay for a service that would let me take a lambda I expect to sit in IO wait for a while and say "hey, I created these sockets, but I don't want to pay for compute while I wait for some network traffic on them, just charge me for the connections".

Concrete Examples:

1. Long Polling: I make a SQL query I know will take at least 120 seconds. I don't want to pay for my lambda to sit around and wait that whole time. I want to be able to "await" (async io terminology) that connection and suspend execution until I get a network response.
2. Outbound Bidirectional Connection Initiation: Amazon Connect has a chat service, the API uses websockets. Facebook has a messenger service, it uses webhooks. I want to be able to proxy facebook messenger messages into amazon connect. To do this I create an APIGW and register my webhook with facebook messenger. Now I receive messages from facebook. Now I want to initiate an outbound websocket connection to amazon connect chat. After I initiate the connection I have to keep it around forever until amazon connect chat closes the connection. I can't stop my lambda execution and invoke it again when I get new traffic. I have no way of offloading it and giving other invocations access to that websocket connection.

There are similar examples for graphql, pub/sub, IoT, and streaming workloads.

Solutions:

Right now most people just use a container for things like this because it's cheaper than having a lambda sit doing nothing but waiting on a network call.

Other solutions involve proxies (either nginx or custom)

Desired Solution:

A service, preferably built into lambda runtime but even external is ok, that manages those connections for me. APIGW/ALB but in reverse. I'm happy to pay for that connection management service as long as it costs less than it would cost to just leave my lambdas doing nothing for minutes at a time.