I think that last sentence is the key: "you just need to find them." You'll work harder to find career opportunities that fit with the distributed lifestyle, but for me it is worth it because of all the non-career opportunity that comes with it.
I've been distributed/remote for almost 10 years and my experience hits all the same points you made. I also have no doubt that my choice has limited my career. But career is just one scope of life. Not commuting to an office has afforded me the time to exercise more, see my children every single day, participate in their lives in ways that would not be possible otherwise, eat healthier, spend time with my aging parents who come to see my children at least once a week, and on and on.
I've been distributed/remote for almost 10 years and my experience hits all the same points you made. I also have no doubt that my choice has limited my career. But career is just one scope of life. Not commuting to an office has afforded me the time to exercise more, see my children every single day, participate in their lives in ways that would not be possible otherwise, eat healthier, spend time with my aging parents who come to see my children at least once a week, and on and on.