Scaramucci to Colbert: 'If it were up to me, Bannon would be gone'