To get reliable message delivery, you have to do that when using WebSockets or SSE too, because those also disconnect or time out depending on upstream network conditions outside the client's control, and will lose messages during reconnection if you don't have a sender-side retransmt queue.
However, queued messages don't have to be kept for a very long time, usually. Because every connection method suffers from this problem, you wouldn't usually architect a system with no resync or reset strategy in the client when reconnection takes so long that it isn't useful to stream every individual message since the last connection.
The client and/or server have a resync timeout, and the server's queue is limited to that timeout, plus margin for various delays.
Once there's a resync strategy implemented, it is often reasonable for rhe server to be able to force a resync early, so it can flush messages queues according to other criteria than a strict timeout. For example memory pressure or server restarts.
With websockets, the client can immediately acknowledge receipt of a message, since the connection is bidirectional. And on a resync the server can just send events that the client never acknowledged.
That (to quote you) means you have to keep them in the queue until the next time the client connects, which could be a very long time.
To be clear, there's no real difference - in both cases you have to keep messages in some queue and potentially resend them until they've been acknowledged.
