Tory minister asks crowd if they want Brexit done, is told 'we don't want it done'