GoGenerator generates code that can not be built if the definition file uses the type of another namespace. I compared four proposals to solve this problem.
- Proposal 1: Add options to
namespace - Proposal 2: Add
base_namespacekeyword - Proposal 3: Refer to GOPATH
- Proposal 4: Use command line flag
All proposals, it has some issues, can solve the problem. And I recommend that use command line flag.
First I checked how Protocol Buffers deals with it.
The Go plug-in of Protocol Buffers does not support multiple pacakge (namespace) in the first place.
Also, the definition of namespace like package foo.bar; will be treated as package foo_bar in Go.
I tried with the following two definition files.
// foo.proto
syntax = "proto3";
package foo;
message Request {
}
==========================
// bar.proto
syntax = "proto3";
import "foo.proto";
package bar;
message BarRequest {
foo.Request request = 1;
string type = 2;
}
C++ generator can interpret these, but Go generator cannot.
$ protoc --version
libprotoc 3.0.0
$ protoc *.proto --cpp_out=cpp
$ protoc *.proto --go_out=plugins:.
2017/02/08 12:41:34 protoc-gen-go: error:inconsistent package names: foo bar
--go_out: protoc-gen-go: Plugin failed with status code 1.
Is it better to match FlatBuffers to Protocol Buffers? I do not think like that.
google/flatbuffers#342 (comment)
prototype: https://github.com/dictav/flatbuffers/tree/fix-go-import-namespace
- It is intuitively configurable
- It can be specified in definition file
- It must be described in every
namespaceand it is cumbersome - Mixing multiple base namespace either generate files that can not be imported or fail to generate (by implementation)
- Confused because of there is the
-ooption of the command line flag as the factor determining the import path - It is hard to treat the same definition file in different projects
First, I considered the two definition files as following:
// protocol.fbs
namespace protocol (go: "github.com/company.com/proj");
table Name {
first: string;
last: string;
}
namespace protocol.user (go: "github.com/company.com/proj");
table User {
name: protocol.Name;
}
Based on these I generated a Go code. Please pay attention to the current directory.
$ pwd
$GOPATH/src/github.com/company.com/proj
$ flatc -g -o flatc_out protocol.fbs
$ tree protocol
flatc_out/protocol/
├── Name.go
└── user
└── User.go
Let's notice the User table that references another namespace type. For example, the generated code will be as follows:
// $GOPATH/src/github.com/company.com/proj/flatc_out/protocol/user/User.go
package user
import (
flatbuffers "github.com/google/flatbuffers/go"
protocol "github.com/company.com/flatc_out/proj/flatc_out/protocol"
)
// ...
func (rcv *User) Name(obj *protocol.Name) *protocol.Name {
o := flatbuffers.UOffsetT(rcv._tab.Offset(4))
if o != 0 {
// ...
An import statement is required to use protocol.Name.
For GoGenerator, it is only this part that namespace is important.
To import protocol.user package in main.go, write as following:
// $GOPATH/src/github.com/company.com/proj/main.go
package main
import (
"github.com/company.com/proj/flatc_out/protocol/user"
)
func main () {
// get User binary
name := user.GetRootAsUser(buf, 0)
// do something
}
It works well.
Next, I changed the files to as following:
// protocol.fbs
namespace protocol (go: "github.com/company.com/proj_base");
table Name {
first: string;
last: string;
}
namespace protocol.user (go: "github.com/company.com/proj");
table User {
name: protocol.Name;
}
Based on these I generated a Go code.
The flatc command and the directory structure are the same as above.
Perhaps User.go will be as following:
// $GOPATH/src/github.com/company.com/flatc_out/proj/flatc_out/protocol/user/User.go
package user
import (
flatbuffers "github.com/google/flatbuffers/go"
protocol "github.com/company.com/proj_base/flatc_out/protocol"
)
// ...
func (rcv *User) Name(obj *protocol.Name) *protocol.Name {
o := flatbuffers.UOffsetT(rcv._tab.Offset(4))
if o != 0 {
// ...
It could not be built.
Please look the line 6.
The import path is github.com/company.com/proj_base/flatc_out/protocol, but actually the protocol package is in $GOPATH/src/github.com/company.com/proj/flatc_out/protocol.
Certainly, we can also write files to $GOPATH/src/github.com/company.com/proj_base/flatc_out/protocol.
However, it is not good to write the files to a different repository than the current project.
To write the files to another project's directory also changes the meaning of flatc -o. I think that all files generated by flatc should be created in the directory specified by -o.
prototype: not implemented
It use the base_namespace keyword.
Please notice that Proposal2 has some of the same problems as Proposal1.
namespace protocol;
base_namespace (go:"github.com/company.com/proj");
- It can be specified in definition file
- It is not necessary to describe in all
namespaces
- If there are more than one
base_namespacedefinition, it is necessary to decide the behavior. Error?, Ignore? Or etc ... - If there are many definition files, it is hard to tell which file has
base_namespace. - It is a little troublesome to treat the same definition file in different projects
prototype: https://github.com/dictav/flatbuffers/tree/fix-go-import-gopath
It compares GOPATH with the output directory and automatically sets the appropriate import path. It worked roughly well.
- No need to change IDL and command line flag
- The range of influence of change is small
- There is no dependency on project (path)
- Users might not set GOPATH. (Users who run
flatc - goare not always user of Go at all times.) - The destination directory is not always the real import path as it is (the user may want to move the code after generating it).
google/flatbuffers#342 (comment)
prototype: https://github.com/dictav/flatbuffers/tree/fix-go-import-cmdflag
Pass base namespace to the command line.
Although it can add a flag like --namespace_prefix, since the treatment of namespace is different for each language, I propose a way to add it as option to language flags as following:
flatc --go="github.com/company.com/proj" \
--java="com.company.proj" \
protocol.fbs
- There is no need to change IDL
- It can uniquely determine the base namespace
- Not dependent on project (path)
- Command line flags become cumbersome
It is possible to make GoGenerator interpret multiple namespace's types in every proposals.
Although there are advantages / disadvantages, we can reduce user confusion by increasing restrictions, increasing error handling and maintaining documents.
For example in Proposal1 and Proposal2, it prohibit multiple base namespace and display as error then it can avoid troubles that do not work during import.
However, I think it is better to use command line flag.
It can tell the users how to use the base namespace with flatc --help.