This is true only when you have relatively steady flows, or infinite multiprocessing. In that case, yes, the queue is in practise either "almost empty" or "infinite".

I think the reason you get the responses you do is people are used to systems with high variation in flow rates that can only process a limited number of items at the same time, and for these, queues can soak up a lot of variability by growing long and then being worked off quickly again.