Two thirds of voters including a large number of Leavers say leaving the EU is contributing to rising supermarket prices and high inflation.