Immigration minister denies placing asylum seekers in barracks has been ‘public health disaster’