Prince Harry 'doesn't have many early memories of Diana'