You don't have to fuck over another one. Historically that is what happens, and it cannot be discussed away that most of Europe's wealth is built on exploitation.
But, for example, why should Sweden be a better place to live than the rest of the EU, or the world? It doesn't even make sense to me to draw these arbitrary lines. What changes when you cross the border to Finland for example?