Ed Miliband and David Lammy discussed concerns that the appointment of Peter Mandelson as the UK’s ambassador to Washington could “blow up”, the energy secretary has revealed.

Miliband said he spoke to Lammy, who was foreign secretary at the time Mandelson was given the Washington post, and both expressed reservations.

Mandelson was sacked after nine months in the job after new disclosures about his relationship with the late financier and child sex offender Jeffrey Epstein.

The Guardian revealed last week that Mandelson had failed his initial security vetting, which had then been overruled by the Foreign Office, leading to the sacking of the department’s permanent secretary, Oliver Robbins.

Robbins will give evidence to a select committee of MPs on Tuesday morning. He is expected to argue he was following procedures by not disclosing details of the vetting process to ministers and by ensuring Mandelson could be appointed, which was then the express desire of No 10.

Mandelson vetting row: Starmer v Robbins blame game deepens - The LatestMandelson vetting row: Starmer v Robbins blame game deepens – The Latest

Miliband told Sky News on Tuesday that he and Lammy had been worried the appointment was risky and said Mandelson should never have been appointed, because of his links with Epstein and his lobbying firm’s business ties with Chinese and Russian companies.

“You’re saying he should never have been appointed and I agree with you,” Miliband told Sky News. “I steered well clear of Peter Mandelson when I became Labour leader in 2010.”

Asked what he had thought about Mandelson’s initial appointment, he said: “That it could blow up, that it could go wrong … I had a conversation with David Lammy about it before the appointment, and I said I was worried about it … I think he was worried about it too.”

Miliband said he was not asked directly what his views were of the appointment. “Maybe I wasn’t the person that people would necessarily ask. I think people knew my view on Peter Mandelson,” he said.

He said he did not think it was a resigning matter for the prime minister. “No, I don’t think he should. Because I think if every time a prime minister made a mistake they resigned, we would shuttle through prime ministers like nobody’s business,” he said.

“Prime ministers make mistakes. I think on big judgments for this country, the biggest judgment of all, whether to join the war against Iran, Keir Starmer made a big and fundamental correct judgment.”

Speaking on BBC Radio 4’s Today programme, the Labour MP Sarah Champion said a leadership challenge was “absolutely the last thing that we want right now” but said Starmer was deeply unpopular.

She said: “I’ll be honest with you, people don’t like Keir on the door but it’s not over this Mandelson thing. They don’t like him personally. There’s been a fantastic campaign by opposition parties to undermine him.”

She said people were more concerned by events in Iran and rising energy costs. “I think that so much attention being given to the minutiae of this just confirms the Westminster bubble in their mind and they don’t like it,” she said.