Just noodling on the future Node.js and http

http-next.js

import http from 'http-next'
import fs from 'fs'

const server = http.createServer({
  allowHTTP1: true,
  allowHTTP2: true,
  allowHTTP3: true,
  key: fs.readFileSync('localhost-privkey.pem'),
  cert: fs.readFileSync('localhost-cert.pem')
})

// Nixed a bunch of error handling to concentrate on the spirit of what I'm getting at here.
// That should totally be added back in. Sorry.
// Now it's just a simple echo server with a favicon since I didn't feel like writing the rest out :D
// Finally: note that I'm not arguing against any other options. Just throwing this out there.

for await (const session of server) {
  ;(async session => {
    for await (const req of session) {
      req.handle(() => {
        // Two main things on this one:
        // 1. All relevant stuff is in a callback which is simply never called in http1.
        // 2. Returnable response objects, which I'll get to in a minute.
        req.serverPush('/favicon.ico', () => {
          const res = new http.ServerResponse()
          fs.createReadStream('./public/favico.ico').pipe(res)
          return res
        }

        const reqBody = await req.body()
        
        const res = new http.ServerResponse(200)
        res.write(reqBody)
        return res // returning here also implicitly ends the writable stream if it has not been ended.
        // Could also support returning buffers or strings
      })
    }
  })(session)
}

// tl;dr i really like the concept of returning a response.

My thinking is, if we always hide that detail away, it might end up that users get unexpected behaviors. If we throw when they try to push and cannot, then they know that can happen and will account for it. Frameworks can and should consider making this simple for users, but also they also typically have a more robust error handling approach to help.

SGTM.

This is one of the things I think will most improve folks basic ability to use the core http apis, so I think the push parts are nice, but I think we just need a good set of built in "serve thing thing" methods (json, files, buffer) were the http stuff is handled for you like when you do it in express.

At first glance, this seems like a lofty goal, because making everyone happy here is going to be tough. Probably worth the effort though.

Now this is an idea I hadn't thought of, but now that you say it, having a .consume method there would be AMAZING!

I'm super curious as to how this would be received in core. There are probably numerous naming collisions in userland (probably also in core itself). You'd also have to support object mode, which I guess would return an array of objects? The interesting bit here, to me, is that a naive implementation would use Buffer.concat, which is kind fo slow, but by being in core, more interesting optimizations could be made.

Anyways, since streams are already async iterators, maybe this would to the trick? https://github.com/tc39/proposal-iterator-helpers

I have a few thoughts on this I would like to present, but I would like to focus on one, the idea of returning a ServerResponse.

I have a library, http-transform, that allows you to do this:

function makeResponse(req){
	const res = new ResponsePassThrough;
	res.statusCode = 200;
	res.setHeader('Content-Type', 'text/plain')
	res.write('Line\r\n');
	// you can also `return res;` but this will return the writable interface, too.
	return res.readable;
}

From here, you can call makeResponse to get an IncomingMessage, and you can pipe it to a ServerResponse object:

function request(req, res){
	// you can also call .pipe() but for compatibility, this doesn't set the headers, it pipes only the body.
	makeResponse(req).pipeMessage(res);
}

Doing this native in Node.js would require a significant undertaking because it has an asymmetrical interface for "Readable" and "Writable" streams. (f the interfaces were symmetrical, you would make data readable by calling write()—but you call push(). Why?)

There are several problems with error propagation I've been unable to completely solve, because of bad assumptions that Node.js makes about event emitters and streams.

To properly implement this, Node.js would need a "Simplex Pair" object, where there is a single buffer that is filled from a Writable side, and drained from a Readable side. (As it stands right now, this is not easy to do, and linking a Readable to a Writable will create at minimum two places where data is buffered.)