The former Brexit secretary has dismissed as ‘propaganda’ frequent references to his comments in 2016 about Brexit having no downsides.