Former Tory leader Lord Howard says he's had a couple of "difficult conversations" with David Cameron but still feels that EU leaders would be reluctant to see Britain leave.