This would work if we were only using them for messaging, but the protobuffer objects are also used as the datastructure in which the information they contain is stored. This was done in the name of simplicity, to avoid creating a separate internal datastructure for each kind of message.
The flow of data is currently:
various external sources -> merged into protocol buffer struct -> later sent to client
To avoid keeping a heap of protocol buffer structs in the heap, this would need to change to:
various external sources -> merged into some internal datastructure -> later converted into protocol buffer struct and sent to client.
> This would work if we were only using them for messaging, but the protobuffer objects are also used as the datastructure in which the information they contain is stored. This was done in the name of simplicity, to avoid creating a separate internal datastructure for each kind of message.
Did you consider keeping strings (serialized messages) in your cache rather than message objects? I do this in C++ code simply for memory efficiency. Here it would also allow you to avoid these extra pointers.
The flow of data is currently:
various external sources -> merged into protocol buffer struct -> later sent to client
To avoid keeping a heap of protocol buffer structs in the heap, this would need to change to:
various external sources -> merged into some internal datastructure -> later converted into protocol buffer struct and sent to client.