Switching measurement systems is tough, but switching number systems would be about a million times harder. (Or about 12^5.5 times harder, if you prefer.) I have trouble imagining where you'd even start....
Because it conflicted with our number system, and we (mostly...) decided that harmony with the number system was better than divisibility. If both were base 12, there wouldn't be any conflict.
