Yeah the backup is really bad... WhatsApp put so much effort into the E2E encryption and then just dumped the backups on Google. It was all just window dressing.
I bet this is also why the intelligence agencies aren't pounding on them for having E2E encryption, obviously Google is an open book to them. And Google has access to all your history for their own purposes.
Of course you can choose to turn off the backups, but you can't be sure all your contacts are doing the same.
The idea of end-to-end encryption is just too broken. You always need to make sure that the implementation of the idea isn't controlled by the vendor from which it is supposed to protect you. But it is pretty much never the case, all the WhatsApps, Signals, etc. control implementations, updates and openly fight against attempts to decentralize or weaken that control, making sure that ultimately they are the ones deciding whether they get access to your data or not. At best it's just acts as a regular encryption with the vendor.
And while there is definitely no hope for binary blob consumer software to ever have real end-to-end encryption, there is hope that it could happen for open source software, distribution of which is not controlled by software vendors with many competing parties that package and ship it and an incentive not to sneak anything in because of that.
> The idea of end-to-end encryption is just too broken. You always need to make sure that the implementation of the idea isn't controlled by the vendor from which it is supposed to protect you. But it is pretty much never the case, all the WhatsApps, Signals, etc. control implementations, updates and openly fight against attempts to decentralize or weaken that control, making sure that ultimately they are the ones deciding whether they get access to your data or not.
Note that the client side apps are available for researchers to study. If they find a back door, it will be a big publicity issue for the app vendor.
Your concern is still valid though. Both Whatsapp and Signal could announce tomorrow that they drop E2EE or add some feature (like a default on non-disable-able cloud based spellchecker) and there is nothing you could do to protect your correspondence.
Matrix is different here that it encourages independent implementations as well as end to end encryption.
I bet this is also why the intelligence agencies aren't pounding on them for having E2E encryption, obviously Google is an open book to them. And Google has access to all your history for their own purposes.
Of course you can choose to turn off the backups, but you can't be sure all your contacts are doing the same.