The figure rises to 80% for businesses that do a moderate or large amount of trade with the EU, a survey has found.