So you want to use GoGo Protobuf

Best practices for using GoGo Protobuf

Introduction

In the Go protobuf ecosystem there are two major implementations to choose from. There’s the official golang/protobuf, which uses reflection to marshal and unmarshal structs, and there’s gogo/protobuf, a third party implementation that leverages type-specific marshalling code for extra performance, and has many cool extensions you can use to customize the generated code. gogo/protobuf has been recommended as the best choice of Go serialization library in a large test of different implementations.

Unfortunately, the design of golang/protobuf and the gRPC ecosystem makes it hard to integrate third party implementations, and there are certain situations where using gogo/protobuf with gRPC can break unexpectedly, at runtime. In this post I will try to cover best practices for working with gogo/protobuf.

TL;DR?

I made an example repo of using gogo/protobuf with various parts of the greater gRPC ecosystem, graciously hosted under the gogo namespace by Walter Schulze, complete with a gRPC-Gateway and OpenAPI UI:

https://github.com/gogo/grpc-example

gRPC-Example repo in action

If you find anything that isn’t listed on there, or in this post, please submit an issue against this repo, and I will attempt to implement a workaround or raise a relevant issue upstream.

Still here? Lets move on to the details.

GoogleAPIs

The google/googleapis and golang/genproto repos provide a large number of protofiles and pre-generated Go files, all maintained by Google’s engineers. However, because they use protoc-gen-go to compile the Go files, they are not strictly compatible with gogo/protobuf, as they do not register with the correct backend.

Instead, if you find you need to reach for these pre-compiled files, use gogo/googleapis. This contains a growing number of Go files pre-generated with protoc-gen-gogo, and registering against the correct backend. If there are any files missing from this repo, make sure to raise an issue (or make a PR) and it’ll be added in no time.

Bonus: because the generated files are in the same folder as the proto files, including the files works with golang/dep, limitations on including non-go files notwithstanding.

Protobuf Any types

The google.protobuf.Any type is used in a wide variety of the GoogleAPIs proto messages, but using it with gogo/protobuf requires extra care. The Any message types work by using the internal “registry” of the protobuf package used, so you need to make sure any messages you stick in an Any container have been generated with gogo/protobuf. Using the gogo/googleapis repo is a great start, but the general rule of thumb is to ensure all protofiles are generated with gogo/protobuf.

gRPC

gRPC is designed to be payload agnostic, and will work out of the box with gogo/protobuf, as while it imports golang/protobuf, it only uses it to type assert incoming interfaces into interfaces that are equally supported by all gogo/protobuf types.

Reflection

gRPC has this cool thing called server reflection, which allows a client to use a gRPC server without having to use the servers protofile, dynamically, at runtime. Some tools such as grpc-ecosystem/polyglot, ktr0731/evans, kazegusuri/grpcurl and fullstorydev/grpcurl (popular pun) have support for dynamic reflection based requests today.

Unfortunately, gogo/protobuf is currently not working perfectly with server reflection, because the grpc-go implementation is very tightly coupled with golang/protobuf. This presents a couple of different scenarios where using gogo/protobuf may or may not work:

  1. If you use just the protoc-gen-gofast generator, which simply generates type specific marshalling and unmarshalling code, you’ll be fine. Of course, using protoc-gen-gofast still comes with downsides, such as having to regenerate the whole proto dependency tree.
  2. If you use protoc-gen-gogo*, unfortunately, reflection will not work on your server. This is because gogo.pb.go does not register itself with golang/protobuf, and reflection recursively resolves all imports, and will complain of gogo.proto not being found.

This is of course quite disappointing, but I’ve discussed with Walter Schulze (the maintainer of gogo/protobuf) how best to solve this and raised an issue against grpc-go. If the maintainers of grpc-go do not want to make it easier to use with gogo/protobuf, there are other alternatives. I’ll update this post once I know more.

gRPC-Gateway

The gRPC-Gateway is another popular project, and at first it might seem completely compatible gogo/protobuf. However, it suffers from a number of incompatibilities, most of which can be traced to its liberal use of golang/protobuf packages directly:

  1. The gRPC-Gateway does not work with gogo/protobuf registered enums.
  2. The default JSON marshaller used by the gRPC-Gateway is unable to marshal non-nullable non-scalar fields.
  3. A bug in the generator means generated files with Well Known Types need post-generation corrections.

Fortunately, workarounds exist for these problems. Using the goproto_registration extension of gogo/protobuf will ensure enum resolution works. Using the gogo/gateway package with the gRPC-Gateway WithMarshaler option fixes the scalar field marshalling issue.

Both of these workarounds are implemented in the gRPC-example repo.

As for the incorrect import, a simple sed post-generation will sort that out (adjust as necessary):

$ sed -i "s/empty.Empty/types.Empty/g" <file.pb.gw.go>

Note that the gRPC-Gateway makes use of google/api/annotations.proto, so make sure you include the correct file from gogo/googleapis as mentioned when compiling your proto files.

Conclusion

Unfortunately, while gogo/protobuf delivers awesome customization options and faster marshalling, getting it working well with the larger gRPC ecosystem is complicated. gogo/protobuf has it as a stated goal to be merged back into golang/protobuf, and recent discussions have been positive, but it’s hard to say whether it’ll lead to anything. There is also an open issue discussing the possibility of type specific marshalling and unmarshalling code in golang/protobuf itself, which is what I think is the biggest reason most users turn to gogo/protobuf.

In a perfect future, we’d have some or all of the customizability and speed of gogo/protobuf with the official backing of golang/protobuf.

I made a repo for experimenting with various go proto generators that you can check out if you want to make your own tests.

If you enjoyed this blog post, have any questions or input, don’t hesitate to contact me on @johanbrandhorst or under jbrandhorst on the Gophers Slack. I’d love to hear your thoughts!

3 comments

Timon Wong
Tuesday, Mar 13, 2018

As a side note, gRPC-Gateway does not work well with github.com/gogo/protobuf/types.FieldMask either here
Reply to Timon Wong

Johan Brandhorst
In reply to Timon Wong
Tuesday, Mar 13, 2018

Ah, yes, thanks for pointing that out, I wasn’t aware of that!

Johan Brandhorst
In reply to Timon Wong
Monday, Apr 2, 2018

I’ve just merged an example of how to successfully use the FieldMask type with gogoproto and the gRPC-Gateway: https://github.com/gogo/grpc-example/commit/6c217371b67a89609c632f047477fa5a1123ac93.
Reply to Thread

Add a comment