The former chancellor said the Conservatives, Labour and national broadcasters "won’t talk about it", but the split with the EU is to blame.