Speaking outside his home, former chancellor Sajid Javid told reporters he was asked to replace all of his political advisers to stay in the role.