I have had similar thoughts as well. Assuming FTL isn’t possible, at some point it wouldn’t make sense to have a cohesive system. Say, there’s an outpost that is 30 years away by the fastest spacecraft.