The reality that Leavers are facing is that the promised benefits of Brexit are hard to achieve while the costs are hard to avoid.