In reality, it is restrictive immigration policy that permits only the best from other nations that creates a "brain drain" where the least-productive workers are left behind in their native country to survive. In an open borders situation, the poorest would be permitted to immigrate, and would be better off.
Many critics of open immigration are concerned that the GDP of the sending countries would reduce, creating a spiral of even more poverty. But this way of thinking only considers countries, and we should be thinking about people.
Aside from the moral problem of treating the wealthy more favorably than the poor, forbidding poor people in other countries from migrating essentially keeps them from making choices that they believe would improve their lives while cooperating with others in a broader economy. Additionally, poor people are often poor because they lack the opportunity to become more skilled and therefore relatively wealthier. The best example of open immigration benefiting not only the immigrants but the receiving areas is the expansion of the United States westward. Not only were free people able to improve their lives and the areas to which they moved, they were free to avoid harsh and unsatisfactory regions because no state was forcing them to remain.
In short, free movement of labor enriches, not impoverishes.
This question is from Faith Seeking Freedom: Updated & Expanded, available in paperback, PDF, and Kindle.