When we think about EU citizens and Brexit, we must factor in former refugees who’ve already faced displacement. Not only is the uncertainty inconvenient, it can be life-threatening.