I am not sure if you understand Federation. It is the States who predominately deliver services in education, health, police, provide infrastructure such as roads/ports/rail and regulate mining for example, so if their is any duplication it is at the Federal level. Now having said that the States are not responsible for Centrelink, Defence, Federal Police and the ATO and others, so duplication is only limited to those functions the State does that is duplicated at Federal level.
If you want to talk about administration, until the latest boom you should have worked in some of the large mining companies in their buildings in Perth who I worked for - you would be surprised at the level of red tape there and inefficiency.
Certainly you can become more efficient and the size of the public service is already been reduced at Fed level.
But public sector bashing is an ideological hobby - public spending in Australia on education is no different than other developed countries (if not less). Public employment is also less than the OECD average and many other countries. I for one don't buy into ideological drivel - I may have in the past but after the number of privatisations in the 80s and 90s of public monopolies, the resulting private monopoly milked the crap out of me and remains completely inefficient (i.e. ring Telstra for assistance for example - you got a better service when it was government owned). Then some of my mates are contractors and they charge double to three times what a public servant gets paid for the same job when contracting to the public sector - they also tell me that they are not that many times as efficient as the public servant they are advising and say it is money for jam.
https://stats.oecd.org/
https://data.oecd.org/eduresource/public-spending-on-education.htm#indicator-chart
