Right, but the point about the closed server infrastructure still stands, though, and you have to blindly trust them not to collect data about you or interfere with your message delivery, compared to a possible decentralized system.
> you have to blindly trust them not to collect data about you
Why blindly trust them if you can inspect the client, like you easily can with Signal? It's all about how you choose to update your client and which due diligence you want to apply, and for this a decentralized system doesn't help.
> This is about the server which is not open source
The Signal server implementation is mostly open-source, and some users managed to run their own instance.
> you wouldn't know if they actually run that code without modifications
That's the point of SGX, but even without it, most guarantees you get from the Signal protocol come from the clients. E.g. you don't need to know what the server is running to ensure that the content of your messages is not leaking. Other than that, being decentralized on its own doesn't magically solve the metadata problem. Signal leaks less metadata than most messengers, including decentralized ones.