pRPC: gRPC on Classic AppEngine today

gRPC is great, but is not available on Classic AppEngine at this time, so while working on the nextgen CI for Chromium we wrote pRPC (provisional RPC) for Go that is compatible with gRPC server/client code, but works on HTTP 1.x and Classic AppEngine. In addition it has gRPC-compatible discovery and a CLI tool.

IDL

Just like gRPC, pRPC uses Protocol Buffers to define API. I assume you know why protobufs are awesome because you care about gRPC.

// helloworld.proto
syntax = "proto3";

package helloworld;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}

This definition is taken from gRPC examples. pRPC proto definition is same as gRPC, but pRPC does not support streams.

Compile .proto

For Go code, we had to slightly alter (in a backward-compatible way) the code generated by protocol buffer compiler, so we wrote cproto tool.

Install cproto:

go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
go get -u github.com/luci/luci-go/tools/cmd/cproto

cproto has minimalistic CLI:

//go:generate cproto

compiles all .proto files in the current directory to .go files
includes $GOPATH/src in proto import path, so you can (and should) import other .proto files with Go-style absolute paths, e.g. import "github.com/user/repo/proto/something.proto";
generates pb.discovery.go file that registers full service description for discoverability. You don't have to worry about it, it is just there and it is optional.
supports only Go

OK, now we have compiled our .proto files to Go code. Let's implement a service!

Service

A service implementation is fully compatible with gRPC:

pRPC uses gRPC codes
HTTP headers are accessible through metadata in the context.

type greeterService struct{}

func (s *greeterService) SayHello(c context.Context, req *helloworld.HelloRequest) (*helloworld.HelloReply, error) {
  if req.Name == "" {
    return nil, grpc.Errorf(codes.InvalidArgument, "Name unspecified")
  }

  return &helloworld.HelloReply{
    Message: "Hello " + req.Name,
  }, nil
}

In the code snippets I omit imports, but you can find them in the source code.

Server

Now let's host the service. Since AppEngine is our target user case, the example is for AppEngine. The init function registers HTTP routers in the default muxer.

// init registers HTTP routes.
func init() {
  // pRPC uses httprouter that implements http.Handler.
  router := httprouter.New()

  // Configure pRPC server.
  var server prpc.Server
  server.CustomAuthenticator = true // Omit authentication. See below.
  helloworld.RegisterGreeterServer(&server, &greeterService{})
  discovery.Enable(&server)
  server.InstallHandlers(router, base)

  // Plug the router into std HTTP stack.
  http.DefaultServeMux.Handle("/", router)
}

// base is the root of the middleware chain.
// This is the place where you can add a hook for all methods
// or configure the context.
func base(h middleware.Handler) httprouter.Handle {
  return func(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
    h(context.Background(), w, r, p)
  }
}

This example registers pRPC HTTP handlers for helloworld service.

I've deployed prpc-helloworld.appspot.com, but you can run your own server locally:

goapp get -u github.com/nodirt/prpc-example/helloworld/server
goapp serve github.com/nodirt/prpc-example/helloworld/server

Assuming your $GOPATH is correctly configured for goapp, this will run a pRPC server locally at port 8080.

CLI client

rpc is a CLI tool that can discover services and call them.

Install rpc:

go get -u github.com/luci/luci-go/client/cmd/rpc

I will use prpc-helloworld.appspot.com for future examples (and so can you), but you can use :8080 for your local server.

Discover services

List available services:

$ rpc show prpc-helloworld.appspot.com
helloworld.Greeter
discovery.Discovery

List service methods:

$ rpc show prpc-helloworld.appspot.com helloworld.Greeter
// The greeting service definition.
service Greeter {
        // Sends a greeting
        rpc SayHello(HelloRequest) returns (HelloReply) {};
}

Describe a service method:

$ rpc show prpc-helloworld.appspot.com helloworld.Greeter.SayHello
// The greeting service definition.
service Greeter {
        // Sends a greeting
        rpc SayHello(HelloRequest) returns (HelloReply) {};
}

// The request message containing the user's name.
message HelloRequest {
        string name = 1;
}

// The response message containing the greetings
message HelloReply {
        string message = 1;
}

Call services

rpc can make RPCs:

$ rpc call prpc-helloworld.appspot.com helloworld.Greeter.SayHello -name $USER
message: "Hello nodir"

Go client

Now let's make RPCs ourselves. A pRPC client implements the same interface as gRPC client, but it is created using a different function:

client := &prpc.Client{Host: "prpc-helloworld.appspot.com"}
greeter := helloworld.NewGreeterPRPCClient(client)
ctx := context.Background()
res, err := greeter.SayHello(ctx, &helloworld.HelloRequest{
  Name: "nodir",
})
if err != nil {
  fmt.Fprintln(os.Stderr, err)
  os.Exit(1)
}
fmt.Println(res.Message)

This may print a gRPC error code to stderr or the greeting to stdout.

Try it yourself:

go get -u github.com/nodirt/prpc-example/helloworld/client/prpchello
prpchello -server prpc-helloworld.appspot.com -name $USER

This should greet you.

gRPC compatibility

Server and client interfaces for servies are same for pRPC and gRPC.
The discovery is just a service that works with both gRPC and pRPC, but it relies on pb.discovery.go files generated by cproto.
The rpc tool currently works with pRPC only, but we will update it when we switch to gRPC. Patches are welcome too.

Authentication

Unlike gRPC, authentication in pRPC is based on HTTP because our main use case is OAuth.

Our repo implements GAE-based OAuth2. To enable it, all you need to do is to import github.com/luci/luci-go/appengine/gaeauth/server package and use our authentication framework, e.g. use CurrentIdentity function to read current user email.

Alternatively, you can set prpc.Server.CustomAuthenticator to true and do whatever you want in base function passed to Server.InstallHandlers.

RPC Explorer.

An HTML page for making RPCs hosted by the server is under development.

nodirt/prpc.md