Conservative voters 'dying out at rate of 2% per year'