I think queues are the wrong abstraction to model business processes. That's why a trivial issue like a non recoverable failure during processing a message becomes such a headache. The same goes for ordering.
An orchestrator like temporal.io allows modeling your business use case using higher level abstractions that hide all this low level complexity.
Disclaimer: I'm the tech lead of the temporal.io open source project and the CEO of the affiliated company.
It is a problem only if you are mixing up application layers.
If you keep your queueing system and business process as separate layers with queueing system serving only as a means of transporting business events then you can make it all to work correctly.
Think in terms of IP protocol (as in TCP/IP). It is unsuitable for transmitting financial transactions. Yet, financial transactions can be made to work on top of it if you separate the layers and treat IP only as a component mechanism of getting data from A to B.
I think we are in agreement here. Temporal does exactly what you described. It uses queues to transporting tasks to processes. But it completely hides them from the business process code.
The issue is that 99.9% of developers use queues directly in their business applications.
Hiding this complexity is useful if it also means handling it. What are the key patterns you apply in temporal to hide it? I’ve had a look at temporal and find it really interesting.
Instead of directly using queues in a Temporal Workflow, the Workflow (which is written with plain code), schedules an Activity that the system is responsible for, behind the scenes, the Activity is just an item put on a queue.
Activities have retry policies which are also handled by the system.
If an Activity attempt fails and should not retry according to the policy, an exception is thrown in the Workflow to be handled using code.
Disclaimer: I'm the tech lead of the temporal.io open source project and the CEO of the affiliated company.