It's just an artifact of 32-bit addressing. Addresses are just an integer from 0 to (2^31 - 1). Using a smaller address space may be possible (although it is still very limited), but dealing with anything other than exactly 32 bits is a TON of extra work, and so it's not reasonably to accomodate what is really a non-issue.
Because the CPU "thinks" in 32-bits, therefore anything not 32-bits needs to be processed first. You would need to first convert your memory address to 32-bits before the CPU could read it, which brings something that would normally be a single CPU clock tick into more than that (it's slightly more complicated than that, but only in ways that make it worse). On top of that, since compilers don't normally do this stuff for you, you would need to end up writing a wrapper to handle every possible memory call, and write it in assembly.
On top of that, you can't use those wasted bits for anything else anyway, so what's the point?
