Tory minister says people 'furious' over 'partygate' but should await investigation