The analogy fails there because our 3manifold space isn't necessarily "embedded" into a higher dimensional construct the way the surface of a sphere is embedded into our 3 dimensional universe. So there isn't necessarily a hyperdimensional up or down to consider.
Or, alternatively, if you think of time as a dimension, then we can call "up" the future and "down" the past. In that case, if you look down inside the past, you'll see that in that analogy the center of the universe corresponds with the big bang! And all the galaxies are equidistant from that point/moment in spacetime.
The surface of the ballon represents a 2d space, where you cannot look "up-down" in the 3rd dimension, just like in our 3d space you cannot look "up-down" in the 4th dimension.
What about looking "up" and "down?" i.e. into the inside of balloon or away from its surface?
I have a hard time wrapping my head around the balloon surface analogy, because galaxies seem to be in all directions of each other..