Andrew Grice on Double Take: I can barely remember a party in such a bad state as the Torys