Would it ever be an actual bottleneck though? If it's not actually impeding throughput, I feel like this is more of an aesthetic argument than a technical one, and one where I'd happily sacrifice aesthetics to make the code simpler.
> some message can exceed 2^32 bytes
Fair enough, but that just makes the question "would 8 bytes per message ever actually be a bottleneck", which I'm still not convinced would ever be the case
I agree that 4 byte overhead in a single item is not a big deal even when the item itself is short. But if you have multiple heterogeneous items, it becomes worthwhile to set an upper bound for the relative ratio of overhead over data. (SQLite, for example, makes extensive use of varints for this reason [1].) I personally think a single byte overhead is worthy for sub-100-byte data, while larger ones should use a fixed size prefix because the overhead is no longer significant.
That makes sense. I've probably been thinking more about remote network protocols, where the time it takes for a message to reach its destination will be so large that the overhead for a few extra bytes per message is negligible; for a protocol used solely within a given local system, there would be a much lower threshold of acceptable overhead.
Would it ever be an actual bottleneck though? If it's not actually impeding throughput, I feel like this is more of an aesthetic argument than a technical one, and one where I'd happily sacrifice aesthetics to make the code simpler.
> some message can exceed 2^32 bytes
Fair enough, but that just makes the question "would 8 bytes per message ever actually be a bottleneck", which I'm still not convinced would ever be the case