Man, I wanted to love gRPC. The disconnect between Protobufs and languages just felt too great though. You could do some weird things that just made every language feel, to some degree, non-idiomatic.
I switched to Twirp at one point to retain the simplicity of RPC + Protobuf, but avoid some of the complexity we didn't need via gRPC... but even that suffered, of course, from the Protobuf problem.
Finally I'm back to plain HTTP and JSON. We don't worry too much about REST fundamentals, and honestly we're more like an ad-hoc (JSON) RPC over HTTP, but it's simple.
The only problem is documentation. The one thing that I found perfect with Protobuf. Seems really hard to have everything here.
But in short, Protobuf is inherently a language of its own. Like JSON or etc. But it's feature rich enough that it can cause a fair number of incompatibilities between a language's preferred style or usage of features.
Where the incompatibility shows up depends on the language. I found it to be very different between Rust and Go, for example.
PREFIX: It's been ~1.5years since I've used Protobuf and Go together, so forgive my memory.
Sure, take Go for example. Protobuf to Go works well, but there are some features of the Protobuf language that just don't exist in Go. Enums, for example. While Go does have constructs that are similar to Enums, they're just different enough to make it a bit weird.
This basic problem gets worse when you try to use it though. The `One of` construct, for example, is sort of impossible in Go. Iirc the Go implementation had to use runtime type checking to give some resemblance to the Protobuf spec.
Rust (which I focus on now) was far better with Protobufs. As far as I remember, there wasn't much of Protobufs that broke idiomatic Rust. However, there was plenty of idiomatic Rust that broke Protobufs iirc. Things like complex data structures behind enum variants. Enum variants as structs, tuple values, etc. iirc it was bad enough that I usually used an abstraction library to write my idiomatic data structures and convert to/from the Protobuf structs.
Which, is how I left it all together. I realized I had a ton of glue code trying to make up for the incompatibilities in Protobufs + (Go|Rust), such that it would just be easier to drop it - at least as far as my code is concerned.
We now struggle with documentation, something Protobuf did excellently, but at least the code smell is gone.
Thanks, that's extremely helpful as I'm working on a language that needs to compile into usable objects. It does have a union type because they wind up being the least common denominator between enums, inheritance, switches, etc. I figured that was going to be tricky to translate into languages without explicit sum types, so I'll definitely take a look at protobuf and Go and see if there's a way to make it fit better.
> The `One of` construct, for example, is sort of impossible in Go. Iirc the Go implementation had to use runtime type checking to give some resemblance to the Protobuf spec.
It looks like they took a reasonable approach[1], and maybe the deeper issue is that protobuf assumes you want direct access to those structures. That's not an unreasonable assumption given its domain, but I can see a red flag: the code generator is solving the problem by generating a forest of types, but that's also something a developer would never do.
Another approach would be to make the oneof implementation more opaque and let you access things via methods. While you'd always want to allow the consumer to ask "which kind of avatar" is this, you could also let the consumer query "get me the avatar image url" and that could return either success or an error.
> I realized I had a ton of glue code trying to make up for the incompatibilities in Protobufs + (Go|Rust), such that it would just be easier to drop it
That's the acid test for whether it works. And it means you can figure out if your language is good by porting a non-trivial codebase using an existing API and see how much glue is required.
I've seen that too, but it's really only a big deal if you're doing something else smelly: Carrying the codegen'd proto objects throughout your application. Better to handle proto messages the same way you should be handling any interface with the outside world: Translate it into your code's own domain model, which can (and should) follow any idiom you like ASAP, and get on with your day.
I think I agree with you, BUT I will say that it's a problem I just don't have with JSON.
I think it's because JSON is inherently smaller, and the spec primarily focuses on basic types that handle data.
With JSON I can write idiomatic code, in any language, and the translation to and from my code is correct. I don't need to abstract away my JSON code for arbitrary reasons, it works.
I'm not sure why that is TBH, I just know that it's a restriction of Protobuf I don't find myself running into with JSON.
