early thoughts on data types for multiselect 2

multiselect-2-sandbox.proto

syntax = "proto2";

// The initiator chooses whether to set up a multiplexed connection, or a "one
// off" stream for a single protocol. Single stream connections may be useful
// for bootstraping a hole-punching protocol through a relay and other things
// where you just want to get "in and out" of a connection asap.
enum ConnectionType {
  MULTIPLEXED = 1;
  SINGLE_STREAM = 2;
}

// First data sent by initiator.
message ConnectionRequest {
  // to distinguish dialer / listener roles in simutaneous connect.
  required bool is_initiator = 1;

  // Whether we want a multiplexed connection or a single stream for one
  // protocol.
  required ConnectionType conn_type = 2;

  // muxers are "special" and listed separately from other supported protocols
  repeated ProtocolId supported_muxers = 3;

  // the initiator can optimistically send the protocol id and optional initial
  // data payload for multiple streams. Hopefully the responder supports one or
  // more of them and will open streams in response using the muxer.
  // For SINGLE_STREAM connections, MUST contain at most one request.
  // If this is empty, the responder won't open any streams, and we assume
  // the initiator will use the muxer to open a stream afterward.
  repeated StreamHeader stream_requests = 4;

  // protocols supported by the initiator (apart from the protos for the
  // requested streams). May be empty or incomplete.
  optional ProtocolTable supported_protos = 5;
}


// A protocol can be identified by either a name or an ephemeral id that's tied
// to a single session. We MAY want to also support static ids for "well known"
// core protos like libp2p's identify, etc. That might get messy as we version
// things though, and third parties might define their own static protocol ids
// that could collide with ours.
message ProtocolId {
  // must be present the first time the given protocol is requested.
  optional string name = 1;

  // core libp2p protocols can have static id's (defined in multicodec table) so
  // we never have to send string names for them. maybe trouble if we're not
  // clear about reserving a range of ids for internal libp2p use.
  optional int64 static_id = 2;

  // ephemeral because the id is valid only for a single connection / session.
  // maybe there's a better name? session_local_id?
  optional int64 ephemeral_id = 3;
}


// A StreamHeader identifies the protocol for a stream, and optionally includes an
// initial data payload.
//
// In the ConnectionRequest, an initiator sends one or more StreamHeaders in the
// `stream_requests` field to ask the responder to open a stream for each of the
// identified protocols. We can't do this with the multiplexer because one
// hasn't been selected yet, but we assume that there _will be_ at least one
// commonly supported muxer, or that we don't care about muxing at all because
// we asked for a SINGLE_STREAM connection.
//
// Asuming we do set up a muxer, every stream opened using the muxer starts with
// a StreamHeader. Essentially we send this instead of the multicodec header in
// multistream-select 1 to route to the correct protocol handler, etc.
message StreamHeader {
  // what protocol to use for the stream
  required ProtocolId protocol = 1;

  // You can eagerly send data for each requested stream. Should have a size
  // limit that's communicated in the API, since the goal is to fit several of
  // these in a single MTU.
  optional bytes initial_payload = 2;

  // If this is true, this new stream is being opened in response to
  // a stream_request sent by the initiator in a ConnectionRequest. Although
  // we're the party actually opening the new stream using the muxer, this
  // stream should be logically associated with the earlier request sent by the
  // other party.
  optional bool is_response = 3;

  // If you support multiple protocols and you want the other party to respond
  // to AT MOST one of them, you can include alternate StreamHeaders in here.
  // Useful for supporting multiple versions concurrently. Ordered according to
  // preference, with least-desirable at the end of the list.
  //
  // Cannot be nested - any `StreamHeader` in this list MUST NOT have further
  // `alternatives`.
  repeated StreamHeader alternatives = 4;
}


// Always the first data sent by the responder (unless they just hang up).
// The primary purpose is to inform the initiator of our chosen multiplexer
// for multiplexed connections.
//
// Also sends some basic status info in case we want to bail and tell the
// initiator why.
message ConnectionResponse {
  // responder informs initiator of chosen multiplexer. MUST be set if the
  // initiator wants a MULTIPLEXED connection, MUST NOT be set for
  // SINGLE_STREAM.
  optional ProtocolId muxer = 1;

  // optionally tell the initator about the protocols we support. If sent,
  // SHOULD include ephemeral ids for protocols that the initiator has told us
  // they support or sent StreamHeaders for.
  optional ProtocolTable protos = 2;

  required Status status = 3;

  optional string error_msg = 4;

  enum Status {
    OK = 1;
    
    ERR_UNKNOWN = 100;
    ERR_NO_SUPPORTED_MUXER = 101;
    ERR_INVALID_REQUEST = 102;
    ERR_I_DONT_LIKE_YOU = 103;
    // etc...
  }
}


// Represents the protocols supported by the sender. If `is_complete` is true,
// this is an exhaustive list, otherwise it's assumed that the peer may support
// more protocols than listed.
message ProtocolTable {
  // These `ProtocolId`s SHOULD include both `name` and `ephemeral_id` to allow
  // the recipient to skip sending names in future `StreamHeader`s
  repeated ProtocolId protos = 1;

  // If true, this is all the protos we know how to speak.
  optional bool is_complete = 2;
}


// TODO: write up some scenarios and see how they play out with these data types
//
// - initiator wants a multiplexed connection, supports yamux and mplex.
//   - optimistically initiates identify and identify/push, and a DHT query by sending StreamHeaders
//   - responder selects mplex in ConnectionResponse
//   - following ConnectionResponse, uses mplex to open two streams
//     - stream for identify response
//     - stream for DHT response
//     - no stream for identify/push, since there's no expected response
//   - each opened stream starts with a StreamHeader with `is_response` set to true
//  
//
//
// - initiator wants a single stream connection, sends fetch request
// - responder sends ConnectionResponse with OK status, followed by StreamHeader for fetch protocol with `is_response` set to true
// - after writing StreamHeader, responder sends fetch response payload and closes connection