That is certainly one problem Windows has. It's also not really orthogonal (I'm looking at you, registry) and doesn't strive for simplicity or elegance like Unix/Linux tend to do. (Why the user folder is so ambiguous and mysterious before Windows 7 is beyond me.)
If Eric Raymond is to be believed, another problem is that it has no unifying concept, a la Unix's "everything is a file" or Apple's "make the GUI simple and focused".
If they had provided APIs for every function that is part of the registry, they could changed the implementation at will. Instead, the registry is just another file system with all the same problems.
Unix also suffers from the same problem of convention over API. They do have a better convention and "everything is a file" is sort of an API onto itself but the concept is pretty much gone once you reach the level of Windowing System.
"Everything is an OLE server" is the closest Microsoft concept. And it is too - files on NTFS filesystems are a lot more than just files, only (nearly) no-one uses all the fancy stuff, and it's not really exposed through Explorer or CMD. You don't really need file extensions, for example, but everyone is so used to them.
The Registry arguably is no more or less sensible than /etc full of cryptically named config files and executable scripts, all in slightly different formats...
The Registry is a pseudo file system inside the file system. It is one of the reasons that Windows is so much harder to backup and restore than Unix is, because parts of it contain system state and other parts contain configuration and the whole is mixed into one big mess.
OK, but it's not as simple as that. Man fstab-sync, for example, and you'll see that your config files can be mysteriously rewritten. Or resolv.conf being dynamically modified by DHCP. /etc is more complicated than just a directory full of text files.
If Eric Raymond is to be believed, another problem is that it has no unifying concept, a la Unix's "everything is a file" or Apple's "make the GUI simple and focused".