The audit released on Tuesday was the first done by the body in four years and covered transport, energy, water, telecommunications and, for the first time, social infrastructure.

The report highlighted the major challenges faced by the country's infrastructure over the next 15 years and will be used to guide Infrastructure Australia's future priority lists.

The audit found housing stress in Tasmania was pushed up by 3.5 percentage points between 2001-04 and 2013-16.

Housing stress is defined as those in the bottom 40 per cent of household income distribution who spend more than 30 per cent on housing costs. The audit noted about nine per cent of Tasmanian households fell into this category

The audit found while incomes nationwide had grown modestly over a decade, growth had been restricted to mainland cities except for Perth.

Incomes had actually dropped in Tasmania, Western Australia, the ACT and Northern Territory.

Infrastructure Australia found population was in decline across rural and remote areas of the country except for Tasmania.

It stated Metro Tasmania had informed auditors it was unlikely to meet legislated public transport accessibility targets by 2022 as significant financial investment was required.

"This has not been provided for in future budgets," the report said.

Labor infrastructure spokesman David O'Byrne said the state budget showed the government planned to spend less on infrastructure next year and even less the year after.

Infrastructure Minister Michael Ferguson said the budget invested $3.6 billion in infrastructure spending.

"Mr O'Byrne is being hypocritical - it wasn't long ago Labor was criticising our budget for spending too much," he said.

Mr Ferguson said reports that compared infrastructure spending across states were "disingenuous" as Tasmania had a much smaller population compared to other states.