I'm not sure these hypotheticals really lead anywhere. Let's take a more stark reductio: is it permissible to weigh one's own interest more heavily than another's? Are we obligated to act in all ways to maximize the total greater good? Is the moral imperative to spend all of our own lives - and every dollar - Bethamically? Should we shoose our romantic partners calculating what would create the greatest sum of happiness in the world, with our mutual happiness only one element? Are we permitted to provide anything for our children beyond the basics before we have used every bit of our personal resources to assist every child in the world has the basics - and then move on incremental improvements in lockstep? And Will - is this really how you live your life?
If we can make any distinction for ourselves as indivviduals, it is beyond me why logically we couldn't weigh the interests of any individual or group to which we belong more heavily than those we do not. Your analysis of the morality of immigration restrictions seems to me more a matter of a tragedy of the commons - the moral satisfaction is personal and outweighs your minor share of any collective cost to one group to which you belong.