Former Labour prime minister Gordon Brown described the Government’s Brexit approach as “a huge act of self-harm”.