gRPC-Web Kong plugin and trailers

gistfile1.txt

### Brief Intro

This answer will be a bit longer than I wanted it to be, but hopefully it clears
things up.

gRPC over HTTP/2 mandates "TE: trailers" (see [1] in references) since it
makes use of trailers - e.g., for the grpc-status and grpc-message headers
(it's useful to send a status at the end of the response rather than too early).

However, many client and server implementations do not fully handle trailers. 
Further, HTTP/2 is not available everywhere. This is where gRPC-Web comes in handy.

gRPC-Web provides an alternative transport for gRPC that is compatible with HTTP 1.1.
In gRPC-Web, the trailers are sent as part of the response body, in chunked
transfer-encoding [2]. (In HTTP/2 are carried in a HEADERS frame that comes after
the last DATA frame carrying the body.)

The example below illustrates that; note the `grpc-status` and `grpc-message` headers in
the response body. In actual HTTP/2 requests, those would be in actual trailers
(more on that later...)

#### Example: HTTP/1.1

==> Request:

```
$ curl -i -X POST localhost:8000/hello.HelloService/SayHello \                              master?
--header 'Content-Type: application/json' \
--data '{"greeting":"kong2.1"}'
HTTP/1.1 200 OK
Date: Thu, 19 Dec 2024 17:55:24 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
X-Kong-Upstream-Latency: 3
X-Kong-Proxy-Latency: 1
Via: 2 kong/3.10.0
Server: kong/3.10.0
X-Kong-Request-Id: 053a81777661691ac590ae5d2563b179

{"reply":"hello kong2.1"}grpc-status: 0
grpc-message: 
```

==> Packet capture:

---- Request

```
Hypertext Transfer Protocol
    POST /hello.HelloService/SayHello HTTP/1.1\r\n
    Host: localhost:8000\r\n
    User-Agent: curl/8.7.1\r\n
    Accept: */*\r\n
    Content-Type: application/json\r\n
    Content-Length: 22\r\n
    \r\n
    [Full request URI: http://localhost:8000/hello.HelloService/SayHello]
    [HTTP request 1/1]
    [Response in frame: 225]
    File Data: 22 bytes
```

---- Response

```
Hypertext Transfer Protocol, has 2 chunks (including last chunk)
    HTTP/1.1 200 OK\r\n
    Date: Thu, 19 Dec 2024 17:55:24 GMT\r\n
    Content-Type: application/json\r\n
    Transfer-Encoding: chunked\r\n
    Connection: keep-alive\r\n
    Access-Control-Allow-Origin: *\r\n
    X-Kong-Upstream-Latency: 3\r\n
    X-Kong-Proxy-Latency: 1\r\n
    Via: 2 kong/3.10.0\r\n
    Server: kong/3.10.0\r\n
    X-Kong-Request-Id: 053a81777661691ac590ae5d2563b179\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.003739000 seconds]
    [Request in frame: 205]
    [Request URI: http://localhost:8000/hello.HelloService/SayHello]
    HTTP chunked response
        Data chunk (25 octets)
        End of chunked encoding
        trailer-part: grpc-status: 0\r\ngrpc-message: \r\n
        \r\n
    File Data: 25 bytes
```

### Example: HTTP/2

Now, let's see what happens if an actual gRPC over HTTP/2 client is used, in
regular gRPC over HTTP/2 proxying (no gRPC-Web plugin involved).

==> Request

$ grpcurl -proto spec/fixtures/grpc/hello.proto -plaintext localhost:10000 hello.HelloService.SayHello
{
  "reply": "hello noname"
}

==> Packet capture:

---- Request

Note the line prefixed with >>>>>>. This shows the user-agent (grpcurl) passed
the `TE: trailers`, as stated in the spec.

Frame 112: 421 bytes on wire (3368 bits), 421 bytes captured (3368 bits) on interface lo0, id 0
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 61403, Dst Port: 15002, Seq: 1, Ack: 1, Len: 365
HyperText Transfer Protocol 2
    Stream: Magic
        Magic: PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n
HyperText Transfer Protocol 2
    Stream: SETTINGS, Stream ID: 0, Length 18
        Length: 18
        Type: SETTINGS (4)
        Flags: 0x00
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .000 0000 0000 0000 0000 0000 0000 0000 = Stream Identifier: 0
        Settings - Header table size : 0
        Settings - Enable PUSH : 0
        Settings - Initial Windows size : 2147483647
HyperText Transfer Protocol 2
    Stream: WINDOW_UPDATE, Stream ID: 0, Length 4
        Length: 4
        Type: WINDOW_UPDATE (8)
        Flags: 0x00
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .000 0000 0000 0000 0000 0000 0000 0000 = Stream Identifier: 0
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .111 1111 1111 1111 0000 0000 0000 0000 = Window Size Increment: 2147418112
        [Connection window size (before): 65535]
        [Connection window size (after): 2147483647]
HyperText Transfer Protocol 2
    Stream: HEADERS, Stream ID: 1, Length 292, POST /hello.HelloService/SayHello
        Length: 292
        Type: HEADERS (1)
        Flags: 0x04, End Headers
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .000 0000 0000 0000 0000 0000 0000 0001 = Stream Identifier: 1
        [Pad Length: 0]
        Header Block Fragment [truncated]: 838644956272d141d7c65a283ee2d9dcc42b1b87eb1968a0ff418ba0e41d139d09b816c0017f00027465864d833505b11f00037669618a129d4f54cc32b840b83f008bf2b4a7b3c0ec90b22d29ec87089d5c0b8170ff008df2b4a7b3c0ec90b22d5d8749ff83
        [Header Length: 495]
        [Header Count: 15]
        Header: :method: POST
        Header: :scheme: http
        Header: :path: /hello.HelloService/SayHello
        Header: :authority: localhost:15002
 >>>>>>>>>       Header: te: trailers
        Header: via: 2 kong/3.10.0
        Header: x-forwarded-for: 127.0.0.1
        Header: x-forwarded-proto: http
        Header: x-forwarded-host: localhost
        Header: x-forwarded-port: 10000
        Header: x-forwarded-path: /hello.HelloService/SayHello
        Header: x-real-ip: 127.0.0.1
        Header: x-kong-request-id: b3cfad5f27669f482eab296978929279
        Header: content-type: application/grpc
        Header: user-agent: grpcurl/v1.8.5 grpc-go/1.37.0
        [Full request URI: http://localhost:15002/hello.HelloService/SayHello]
        [Response in frame: 143]

---- Response

Frame 143: 142 bytes on wire (1136 bits), 142 bytes captured (1136 bits) on interface lo0, id 0
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 15002, Dst Port: 61403, Seq: 55, Ack: 406, Len: 86
HyperText Transfer Protocol 2
    Stream: HEADERS, Stream ID: 1, Length 16, 200 OK
        Length: 16
        Type: HEADERS (1)
        Flags: 0x04, End Headers
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .000 0000 0000 0000 0000 0000 0000 0001 = Stream Identifier: 1
        [Pad Length: 0]
        Header Block Fragment: 20880f108b1d75d0620d263d4c4d6564
        [Header Length: 54]
        [Header Count: 3]
        Header table size update
        Header: :status: 200 OK
        Header: content-type: application/grpc
        [Time since request: 0.010318000 seconds]
        [Request in frame: 112]
HyperText Transfer Protocol 2
    Stream: DATA, Stream ID: 1, Length 19
        Length: 19
        Type: DATA (0)
        Flags: 0x00
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .000 0000 0000 0000 0000 0000 0000 0001 = Stream Identifier: 1
        [Pad Length: 0]
        DATA payload (19 bytes)
        [Connection window size (before): 2147483647]
        [Connection window size (after): 2147483628]
        [Stream window size (before): 2147483647]
        [Stream window size (after): 2147483628]

vvvvvvvvvvvvvvv

GRPC Message: /hello.HelloService/SayHello, Response
Protocol Buffers: /hello.HelloService/SayHello,response
HyperText Transfer Protocol 2
    Stream: HEADERS, Stream ID: 1, Length 24
        Length: 24
        Type: HEADERS (1)
        Flags: 0x05, End Headers, End Stream
        0... .... .... .... .... .... .... .... = Reserved: 0x0
        .000 0000 0000 0000 0000 0000 0000 0001 = Stream Identifier: 1
        [Pad Length: 0]
        Header Block Fragment: 00889acac8b21234da8f013000899acac8b5254207317f00
        [Header Length: 40]
        [Header Count: 2]
        Header: grpc-status: 0
        Header: grpc-message: 
        [Time since request: 0.010318000 seconds]
        [Request in frame: 112]

^^^^^^^^^^^^^^^

Note that the `HEADERS` frame was indeed sent as a trailer, after the body
(`DATA` frame).

### Conclusion

The gRPC-Web plugin is indeed behaving as expected: it maps a gRPC request
onto an HTTP/1.1 request, where trailers are sent as part of the body. Therefore,
the `TE: trailers` header isn't used.

Nginx, via its `ngx_http_grpc` module, which we use to proxy gRPC requests, proxies
the TE header if and only if the original request had a `TE: trailers` set.

### References

[1]: https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md
[2]: https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1