They haven’t been empty for two years and my waiting times average 5 minutes and 30 minutes during high load times. I don’t understand how they will grow to infinity.
> I don’t understand how they will grow to infinity.
You either have to be consuming faster than the queue is growing, or you are adding faster than you consume. There is no equilibrium.
If you are consuming faster than it is growing, at some point you'll consume all jobs, meaning your queue is sometimes empty. If you are adding faster than you consume, your wait time approaches infinity (tho in reality you'll fill up your queue storage first, of course).
That is only the case if the consumption rate is fixed. If the rate of consumption is adjusted in a feedback loop based on queue depth and/or latency then clearly you can keep your queues in a balance of being neither empty nor infinite.
No, it still applies with variable computing. All you're doing is adding a N length or X second buffer to your queue. You are just shifting your "zero" over by N items or X seconds, which is the time you wait until you scale up.
You may need that or want to add this buffer - but you're not solving any computation resource problem directly. If your jobs take XYZ CPU cycles they take XYZ CPU cycles - it doesn't matter if you make them wait a little bit.
You could even be spending more money, since it does take some resources and cost to scale things.
It doesn't mean it's not valid to do things this way, I'm just saying it's still the exact same problem.
