Senator Gallagher and Treasurer Jim Chalmers were sworn in last Monday, while taking on a swag of interim ministries, so the government could hit the ground running on its key priorities.

That includes getting the budget back in order and in preparation for Labor's first budget in October.

Dr Chalmers will also provide an economic update when parliament resumes in June or July.

"Clearly we are inheriting a very serious set of economic and budget challenges, and we don't want to pretend it is anything but that," Senator Gallagher told ABC's Insiders program on Sunday.

She says there are budget pressures coming in terms of areas such as health, aged care, the NDIS, defence and national security, all of which are growing faster than the economy.

But Senator Gallagher said Prime Minister Anthony Albanese had also committed to the budget having a "cost-of-living lens" applied to it to see what was possible.

But she again said it was unlikely that the halving of fuel excise introduced by the previous government would be extended beyond September, saying it comes in at around $3 billion for a six-month period.

Mr Albanese told Sky New's Sunday Agenda program that his economic ministers had already started an audit on the budget, going through it line-by-line to search out the waste and rorts.

"Where there is waste that we can act upon, we certainly will do so," Mr Albanese said.

"This (previous) government abused the system of the contingency reserve to create these pots of money that were there just to be used for political purposes and we need to do better than that."

Independent MP Zali Steggall agreed there needed to be fiscal discipline.

"We simply cannot continue rorting our way through the public purse," she told Sky News.

Mr Albanese again ruled out ditching the so-called 'stage three' tax cuts legislated to start in 2024/25.

He said Labor was committed to delivering what had been legislated and people were entitled to operate on the basis of the certainty.

Senator Gallagher also backed her leader in not copying a British government windfall tax on the profits of oil and gas companies benefiting from a global price spike, and using that revenue to fund cost-of-living relief.

"We are going down a different pathway," she said.

Australian Associated Press