“Clerks are never going to be able to have this role at the Court that they have now,” the Fox host predicted