> all the popular server distros (I guess not CoreOS, but that's pretty new) work just fine as desktops

CoreOS is basically ChromiumOS minus the GUI. It'd be pretty easy to make a "desktop CoreOS" distribution by adding ChromiumOS components.