The UK is poorer than it would have been, partly due to the war in Ukraine, but also the pandemic, Levelling Up Secretary Michael Gove has admitted.
But he said ministers were taking action on the soaring cost of living, including giving help on energy bills.
The head of the independent forecaster, the Office for Budget Responsibility (OBR), said living standards were seeing their biggest squeeze on record.
Richard Hughes said Brexit had been similar to the pandemic in its impact.
"It's a shock to the UK economy of the order of magnitude to other shocks that we've seen from the pandemic, from the energy crisis," he told the BBC.
Poor productivity had also hurt growth, he said.
And he warned living standards would not return to pre-pandemic levels for another five to six years.
Asked whether he agreed with the OBR's assessments, Mr Gove said economic forecasting was "a very difficult exercise". He added that the UK was dealing with "the aftershocks of two significant events".
"[There's] both the war in Ukraine, the first time we've had war on this scale on the continent in Europe since the Second World War, and the Covid
pandemic, the biggest global health pandemic since the end of the First World War," he told the BBC's Sunday with Laura Kuenssberg programme.
"They have had a huge effect on our economy and on others' economies."
Mr Gove denied that the government was to blame after 13 years in power, but added: "One can always do better, yes."
However, he insisted ministers were taking action to address soaring inflation - the rate at which prices rise - by taxing oil and gas firms' profits and lowering household energy bills.
He also said the Budget had taken steps to help people back to work and to help families, including with childcare.
The OBR forecasts that inflation will fall below 3% this year - down from 10.4% currently - as food and energy prices rise less quickly.
But speaking on the same programme, OBR chairman Richard Hughes said the outlook was volatile given Britain was a net importer of food and energy - the prices of which were set in global markets.
Mr Hughes added that the longer term outlook for the economy was bleak, with people's real spending power - allowing for inflation - not forecast to recover to pre-pandemic levels until the end of the decade.
Mr Hughes blamed a range of issues for holding back the economy, saying: "We've lost around 500,000 people from the labour force, we've seen stagnant investment since 2016 and also our productivity has slowed dramatically since the financial crisis and not really recovered."
He also said that overall output was forecast to be 4% lower than it would have been as a result of leaving the EU.
Last week the Bank of England put up interest rates for the 11th time since December 2021 as it continued its battle to ease inflation.
The decision to lift rates to 4.25% from 4% came after the inflation rate rose unexpectedly last month to 10.4%.