Brexit is hurting the UK economy, Bank of England officials said Wednesday, even as government leaders downplay the impact of the seismic EU withdrawal.