In his speech, Mr Welby told the House of Lords a head had to roll after the publication of the review into the handling of the scandal.
The speech prompted a furious backlash from abuse victims who said it displayed no remorse for the suffering of survivors and said Welby’s delivery of jokes had struck a “frivolous” tone.
“The reality is that there comes a time if you are technically leading a particular institution or area of responsibility where the shame of what has gone wrong – whether one is personally responsible or not – must require a head to roll,” the archbishop said in the Lords on Thursday.
“And there is only, in this case, one head that rolls well enough.”
He also referred to a 14th century predecessor who had been beheaded, adding: “I hope not literally.”
On Thursday, Mark Stibbe, who has previously told the BBC he was groomed and beaten by Smyth in the 1970s, said the archbishop’s joke about “one head” rolling was “disturbing”.
“Smyth survivors want all those responsible to stand down,” Mr Stibbe added.
Another of Smyth’s victims, given the pseudonym Graham Jones in the Makin report, said Mr Welby had got the tone of his speech “entirely wrong”.
In a statement on Friday, the archbishop said he wanted to “apologise wholeheartedly” for the hurt he had caused.
“I understand that my words – the things that I said, and those I omitted to say – have caused further distress for those who were traumatised, and continue to be harmed, by John Smyth’s heinous abuse, and by the far reaching effects of other perpetrators of abuse,” the statement said.
“It did not intend to overlook the experience of survivors, or to make light of the situation – and I am very sorry for having done so.
“It remains the case that I take both personal and institutional responsibility for the long and retraumatising period after 2013, and the harm that this has caused survivors.
He concluded: “I continue to feel a profound sense of shame at the Church of England’s historic safeguarding failures.”
Joanne Grenfell, Julie Conalty and Robert Springett, Church of England lead bishops for safeguarding, wrote to some abuse survivors after Mr Welby’s speech, calling it “mistaken and wrong”.
“Both in content and delivery, the speech was utterly insensitive, lacked any focus on victims and survivors of abuse, especially those affected by John Smyth, and made light of the events surrounding the Archbishop’s resignation,” the letter said.
“It was mistaken and wrong. We acknowledge and deeply regret that this has caused further harm to you in an already distressing situation.”
The letter said the church had “seriously failed” to meet its safeguarding obligations “over many years” and described Mr Welby’s speech as “the antithesis of all that we are now trying to work towards in terms of culture change and redress with all of you”.
Mr Welby will step down on 6 January, with the Archbishop of York, Stephen Cottrell, taking charge until a permanent replacement is found – a search expected to take around six months.
His resignation follows the publication of the Makin report, which said Smyth’s abuse had been covered up by the Church of England for decades.
A barrister and preacher, Smyth is believed to have abused more than 100 boys and young men at Christian summer camps in England in the 1970s and 1980s, and later in South Africa and Zimbabwe.
He is thought to have continued his abuse until 2018 when he died in Cape Town, aged 75.
The independent review said Church officials, including Mr Welby, “could and should” have reported Smyth to the police and authorities in South Africa in 2013.
Mr Welby said he was “told the police had been notified” in 2013 and “believed wrongly that an appropriate resolution would follow.“
