I’m not sure this is the blanket case. Hot air balloons, for example, get right-of-way regardless, on the assumption they have less maneuverability. Weird edge case, I know, but just throwing it out there to underscore the danger of absolute statements.
I have to admit I have not piloted a ballon but even a sailing vessel over taking a power boat has to avoid the vessel in front. I would also question the rationality of operating an aircraft you cannot steer:)
But who gets the right of way between 2 hot air balloons?
Funny story, I actually "crashed" in a hot air balloon as a kid, when the hill we were landing on had a draft running up over it that caught the balloon after the basket had touched down, and dragged us along the ground sideways for a good quarter mile.
If I recall correctly, the lower one has right of way because visibility from lower to higher is blocked compared to higher to lower. It is easier for the one further up in the air column to spot and react to the lower one. Going up is also likely a less dangerous proposition than going down.
This is half remembered from a Snowmass balloon rally conversation.