Vigilantism has flourished due to lack of deference - Minister