David Cameron attacked over refugee vote