I suspect the explanation is similar to the explanation of the abundance of people who seem to have no problem with US paper standards whatsoever, namely, that it's just paper, and most people don't have particularly strict usage constraints, so they just use what they have and it works fine.
No. Try to layout a document on A4 while maintaining at least basic standard rules (e.g. lines of 8-12 words, a pretty text body layout, e.g. a "nice ratio" like 3/4 or 1/2, with standard margins, that is 1 on top, left and right and 2 on bottom). Since your font size will either be 11 or 12 pt on A4 because otherwise you use less than half of the paper or your font is really huge, you're not left with many options.
I wouldn't be so hasty to dismiss hundreds, perhaps thousands of years of aesthetic tweaking by experts as voodoo simply to defend a page size and proportion whose primary advantage is not aesthetic but function.
No units because its all about proportions. E.g. the bottom margin is "double" so it can accommodate page numbers and set the text body on the "horizon".
This is puzzling - how do you explain the abundance of people who seem to have no problem with this whatsoever?
Finally: I have seen lots and lots of A4 books, but they're simply not as common. It probably does help to live in Europe in that regard.