Defence Secretary Ben Wallace has been slapped down by Number 10 after being filmed in an unguarded moment discussing the reasons for proroguing Parliament.