Update on Go 1.24 tool dependencies Go 1.24 introduced the -tool flag to go get and the tool stanza to go.mod. This makes tool dependencies an excellent way to manage protobuf plugins. I have rewritten this post with that in mind. Background When working with Protobuf and gRPC, one of the first things that you as a user has to figure out is how to generate code from your Protobuf definitions....
Introduction In the Go protobuf ecosystem there are two major implementations to choose from. There’s the official golang/protobuf, which uses reflection to marshal and unmarshal structs, and there’s gogo/protobuf, a third party implementation that leverages type-specific marshalling code for extra performance, and has many cool extensions you can use to customize the generated code. gogo/protobuf has been recommended as the best choice of Go serialization library in a large test of different implementations....
One of the gotchas of using gRPC is that it was not designed to transport large messages in one chunk. The default max message size is slightly arbitrarily set at 4MB today, and while it is possible to configure, that kind of behaviour might lead to a slippery slope scenario of ever increasing max message sizes. So what do we do when the message size is too large? We chunk the data into smaller pieces and stream it, using the gRPC streaming methods, naturally....
In a previous blog series I’ve talked about how to work with a gRPC backend from the GopherJS world. It relies on the gRPC-gateway which is a great piece of tech, but unfortunately carries a couple of downsides: Clients don’t know what types are used - the interface is HTTP JSON. This can be somewhat mitigated with the use of swagger generated interfaces, but it’s still not perfect. The interface being JSON means marshalling and unmarshalling can become a significant part of the latency between the client and the server....
I’ve had my fair share of dealing with proto files in go (and to some extent JS), so I thought I’d share some stuff I’ve learnt the hard way by working with proto files. Protoc include paths The protoc include paths can be pretty confusing, so I’ll give a few examples of how to use it properly. Just include the current directory protoc requires that the files referenced are in the include path, so if you’re referencing files relative to the current directory, you’ll need to specify -I....
Putting it all together As we touched upon earlier, we generate a package from the generated JS (meta, right?), which can be served from the server. We’ll create a new file, main.go, in which we can spin up a server, serve the frontend and the gRPC backend. Much of the logic in this file is inspired by the excellent blog post put together by Brandon Philips over at CoreOS. We’re using his elegant solution to serve both the HTTP API and the gRPC API on the same port....
Implement the client GopherJS can be used in a couple of different ways, and there’s a couple of different bindings to many popular JavaScript frameworks. The GopherJS wiki has some great resources. I tried a couple of different ones and ended up using the VueJS bindings because it made it easy to prototype things quickly. I hear VueJS works well for many JS evangelisers out there, but I’ve only used it with small projects....
Implement the server I like to start by creating a struct and write a simple definition that’ll immediately fail to compile. package server import ( "github.com/johanbrandhorst/gopherjs-grpc-websocket/protos/server" ) type Server struct{} var _ server.MyServerServer = &Server{} This won’t compile, because the Server struct does not implement the server.MyServerServer interface. But it’ll also tell us what we’ve got left to implement. So lets implement the server methods: func (s Server) Simple(ctx context.Context, _ *empty....
Create the protobuf interface We’ll start by creating a folder for the package we want to create. Lets call it server. Next we type up a .proto file to define the interface between the server and the client. We’ll have to include some extra proto annotations in order to have the gRPC-gateway generate the translations methods we need. Lets define a simple service with a single HTTP GET endpoint: service MyServer { rpc Simple(google....
I’ve been using gRPC and Go a lot in the last year. At Cognitive Logic every one of our backend services is implemented with Go and gRPC, and it enables us to abstract away most of the complexities of networked micro services and keep interfaces typed and well defined using Google protobuffers. I really enjoy using both, but sometimes I need to write a frontend to a web server and I despise writing Javascript....