Brexit minister Steve Baker says civil servants may be distorting Brexit impact evidence