Not at all. We still have election in this country. When the voters feel Labor is not up to the job of governing the country, I am sure they will express their views in the ballot box whenever the next election is called.
"the biggest hurdle", I would suggest, is the one faced by the Liberals/Nationals coalition opposition in convincing the public that they deserve to be back on the treasury bench.