It's a great idea from a consumer & aesthetic POV, but the technical challenges would have been interesting.

It's also a reinvention of one-bus-for-different-boxes computing from the 70s - Unibus, Massbus, and so on.

It could have been made to work, but it would have been speed-limited, and mechanically unreliable if it wasn't done exactly right.