r5d.2xlarge

Legend
===================
Methods:
  fd:      client sends a file descriptor for the server to fill
  s-write: server writes data the passed fd using write() syscall
  s-mmap:  server mmaps() the passed fd and writes using memset()
  s-reuse: server holds on to the mmapped region across calls
  c-reuse: client holds on to the mmapped region across calls
  sidecar: server sends back data over unix socket
  nt:      server uses nontemporal store instructions to write

NOTE: s-reuse without c-reuse isn't very effective since the server
will detect that it cant actually reuse the mapped buffer, since the
client provided a new one.
Columns designate which core each thread is bound to:
  client: the client 'user' thread
  c-rx:   the client reactor thread
  s-rx:   the server reactor thread
  worker: the server RPC worker thread

Core notation:
  SaCbTc: socket a core b thread c

Speed-limit (optimal) performance: 13858.3 MB/sec
                        method client   c-rx   s-rx worker	MB/sec
# all-same-core
                   fd(s-write) S0C0T0 S0C0T0 S0C0T0 S0C0T0	2093.0
                    fd(s-mmap) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1511.9
                 fd(s-mmap,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1174.2
            fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1486.2
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	1162.3
           fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	3942.5
            fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	4346.0
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	2871.2
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C0T0 S0C0T0	8960.3
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C0T0 S0C0T0	4670.8
                       sidecar S0C0T0 S0C0T0 S0C0T0 S0C0T0	3481.9
                    sidecar,nt S0C0T0 S0C0T0 S0C0T0 S0C0T0	2267.9
# client-server-cross-core
                   fd(s-write) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1695.4
                    fd(s-mmap) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1339.4
                 fd(s-mmap,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1093.4
            fd(s-mmap,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1378.7
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	1069.8
           fd(s-write,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	3068.3
            fd(s-mmap,c-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	3794.0
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	2754.1
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C0T0 S0C1T0 S0C1T0	7050.1
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C0T0 S0C1T0 S0C1T0	4647.3
                       sidecar S0C0T0 S0C0T0 S0C1T0 S0C1T0	2363.9
                    sidecar,nt S0C0T0 S0C0T0 S0C1T0 S0C1T0	1820.9
# client-server-cross-socket
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0414 16:07:46.344928  6799 rpc_stub-test.cc:1094] Local system doesn't have appropriate CPU topology to execute test scenario client-server-cross-socket
# every-hop-crosses-core
                   fd(s-write) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1681.8
                    fd(s-mmap) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1330.3
                 fd(s-mmap,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1079.6
            fd(s-mmap,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1358.3
         fd(s-mmap,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	1107.0
           fd(s-write,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	3005.4
            fd(s-mmap,c-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	3685.9
         fd(s-mmap,c-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	2680.4
    fd(s-mmap,c-reuse,s-reuse) S0C0T0 S0C1T0 S0C2T0 S0C3T0	6509.0
 fd(s-mmap,c-reuse,s-reuse,nt) S0C0T0 S0C1T0 S0C2T0 S0C3T0	4496.5
                       sidecar S0C0T0 S0C1T0 S0C2T0 S0C3T0	2263.5
                    sidecar,nt S0C0T0 S0C1T0 S0C2T0 S0C3T0	1835.9
# every-hop-crosses-socket