Addition/subtraction are also much simpler/cheaper than they would be in an entirely logarithmic model. If floats were just 2^x with some 64 bit fixed point x, it's not clear to me how to do addition efficiently.
The mantissa is linear. It's unrelated to how positional number systems work. A floating point value is split into two numbers - the exponent and the mantissa. Normally they are used to represent a final number like:
x = 2^e * (1 + m)
Where e is the exponent and m is the mantissa (varying linearly from 0 to 1).
But you could have a fully exponential number format:
x = 2^(m + o)
As pointed out though, it makes addition much more complicated, you can't exactly represent integers, and someone told me it makes quantisation noise worse too. Bad idea.
2015.000000000000: https://news.ycombinator.com/item?id=10558871