Correct, because in doing so you max freedom, growth and power, which you don't want to stunt in order to avoid a made up concern that may or may not happen and can be prevented before it happens if it almost happens. Anyway, the market will tend to equilibrate. If one country hypothetically gets "overpopulated", then the less populated place will become more attractive (although I strongly challenge this, nobody wants to move to rural areas within so-called overpopulated countries -- overpopulation is a myth by anti-freedom conservatives), leading to reverse migration.
I'm making practical and utility arguments here but I also simply think noone has the right to prevent someone's freedom of movement unless the stakes are truly dire (infectious disease with high mortality). I believe it's evil for people whose ancestors immigrated (which is everyone, including England, which used to be black inhabited) to then point guns at the border and prevent others from doing so for no reason whatsoever aside from fear and bigotry masquerading as vacuous excuses. The planet is so small, it's a heinous assault of freedom to deprive humans of the freedom to explore and settle around it during our brief existence.
I think there is value in the participants of a society, principally in the governance of a society, being educated and adhering to a certain set of common principles and sharing some level of culture.
Due to the lack of a good alternative, we allow anyone born in this country to be a citizen, and to vote at age 18. I wouldn't want random people who don't speak the language nor have any interest in sharing our culture or respect for human rights or democracy to have political power. (See: the importance of public education for a well-run society).
So let's say we allow an unlimited number of people in with almost no barrier, who may or may not share our political values or language, to live and work here. Sure, it's happened before in mass immigration movements. What if we said, "You can be here, but you can't vote until you are documented to have lived here for a certain period of time and can pass a citizenship test?"
That may work, but I wonder what will happen if we continue to have (as we now do) a two-tiered system ala Starship Troopers; one of citizens and one of "civilians" who do do not have political power. What are the impacts? I am not sure.
My core belief is that education and common values are important to a society, and I am concerned that unfettered immigration into this country would risk destroying whatever common culture/language we have over time.