Brexit is dragging down the economy and its impact is set to get worse due to new trade barriers, the Government’s Budget watchdog is warning.