Security Minister says 'action can be taken' if unlawful parties held