You're on your own if you refuse to be evacuated - Minister