Protocol Buffer CheatSheet

proto3.md

Proto3 Cheat Sheet

Information From: https://developers.google.com/protocol-buffers/docs/proto3

A few rules

• Declaring Message In Protocol Buffer:

• As you can see, each field in the message definition has a unique number.

• Field numbers identify fields in message binary format. Should not be changed once message is in use

• Field numbers between 1 and 15 take one byte to encode, incl. field number and type

• Field numbers between 16 and 2047 take two bytes. Reserve 1...15 for frequently occurring elements.

• Leave some room for possible future frequently occurring elements.

• Field numbers range from 1 to 536,870,911.

• Numbers 19000...19999 are reserved and the compiler will complain

• You also can't use any previously reserved field numbers.

Syntax For Declaring Message

During use, empty fields will be assigned default values

message ${MessageName} {
	${Scalar Value Type} ${FieldName1} = ${Tag Number1};
	...
	${Scalar Value Type} ${FieldNameN} = ${Tag NumberN};
}

Example

syntax = "proto3";

message MessageTypes {

	   // Scalar Value Types
    string stringType = 1; // A string must always contain UTF-8 encoded or 7-bit ASCII text. Default value = ""



    // Number Types, Default Value = 0

    int32 int32Type = 2; // Uses Variable Length Encoding. Inefficient For Negative Numbers, Instead Use sint32.

    int64 int64Type = 3; // Uses Variable Length Encoding. Inefficient For Negative Numbers, Instead Use sint64.

    uint32 uInt32Type = 4; // Uses Variable Length Encoding

    uint64 uInt64Type = 5; // Uses Variable Length Encoding

    sint32 sInt32Type = 6; // Efficient in encoding for negative numbers. Use instead of int32 for negative numbers

    sint64 sInt64Type = 7; // Efficient in encoding for negative numbers. Use instead of int64 for negative numbers

    fixed32 fixed32Type = 8; // Always four bytes. More efficient than uint32 if values are often greater than 2^28.

    fixed64 fixed64Type = 9; // Always eight bytes. More efficient than uint64 if values are often greater than 2^56

    sfixed32 sfixed32Type = 10; // Always four bytes.

    sfixed64 sfixed64Type = 11; // Always Eight bytes.

    double doubleType = 14;

    float floatType = 15;

    bool boolType = 12; // Boolean Type. Default Value = false

    bytes bytesType = 13; // May contain any arbitrary sequence of bytes. Default Value = Empty Bytes



    enum Week {

        UNDEFINED = 0; // Tag 0 is always used as default in case of enum

        SUNDAY = 1;

        MONDAY = 2;

        TUESDAY = 3;

        WEDNESDAY = 4;

        THURSDAY = 5;

        FRIDAY = 6;

        SATURDAY = 7;

    }

    Week wkDayType = 16;



    // Collection of scalars syntax: repeated ${ScalarType} ${name} = TagValue
    repeated string listOfString = 17; // List[String]

}

message FileChunk {
   optional int64 offset = 1;
   optional bytes data = 2;
}

Defining Message Types from Other Message Definitions

message Person {
    string fname = 1;
    string sname = 2;
}

message City {
    Person p = 1;
}

Nested Message Definitions

message NestedMessages {

    message FirstLevelNestedMessage {

        string firstString = 1;

        message SecondLevelNestedMessage {

            string secondString = 2;

        }

    }

    FirstLevelNestedMessage msg = 1;

    FirstLevelNestedMessage.SecondLevelNestedMessage msg2 = 2;

}

Importing Message From A File

one.proto

message One {
	string oneMsg = 1;
}

two.proto

import "myproject/one.proto"

message Two {
	string twoMsg = 2;
}

Advanced Topics

Handling Message Type Changes:

• Never Change/Use The TagNumber Of A Message Field Which Was Removed

• We should use reserved in case of message definition update.

• See https://developers.google.com/protocol-buffers/docs/proto3#updating

Reserved Fields

• It's used in case if we need to add/remove new fields into message.

• Using Reserved Backward and Forward Compatibility Of Messages can be achieved

message ReservedMessage {

    reserved 0, 1, 2, 3 to 10; // Set Of Tag Numbers Which Can't be reused.

    reserved "firstMsg", "secondMsg", "thirdMsg"; // Set Of Labels Which Can't Be reused.

}

Any

• The Any message type lets you use messages as embedded types without having their .proto definition.

• An Any contains an arbitrary serialized message as bytes,

• along with a URL that acts as a globally unique identifier for and resolves to that message's type.

• For Any to work we need to import it as shown below.

    import "google/protobuf/any.proto";

    message AnySampleMessage {

        repeated google.protobuf.Any.details = 1;

    }

OneOf

• There are cases, wherein only one field at-most might be present as part of the message.

• OneOf messages can't be repeated.

message OneOfMessage {
    oneof msg {
        string fname = 1;
        string sname = 2;
    };
}

Maps

• Map fields cannot be repeated.

• Ordering Of A Map Is Not Guaranteed.

message MessageWithMaps {
    map<string, string> mapOfMessages = 1;
}

Packages

• Used for preventing name clashes between protocol message types

• Syntax:

package ${packageName};

• To Access the package;

${packageName}.${messageName} = ${tagNumber};

Services

• Message Types Defined For Using In RPC system.

• When protoc compiler generates for various languages it generates stub methods for the services.

message SearchRequest {
    string queryString = 1;
}

message SearchResponse {
    string queryResponse = 1;
}

service SearchService {
    rpc Search (SearchRequest) returns (SearchResponse);
}

Generating Classes In Go For Protocol Buffers

protoc --proto_path=IMPORT_PATH --go_out=DST_DIR path/to/file.proto