"Why is it everybody I talk to with out exception thinks we, that is Australia, are going backwards?"
The answer is pretty simple. Your "everybody" is either a Liberal supporter or a Liberal-leaning person.
I probably mix with a more balanced group of people: I had come across people who are dead set against Labor and everything Labor does is wrong or no good, and I had also come across people who felt this Labor government had done a great job in stirring Australia through the GFC and is continuing to have a good handle on the economy which has historically low unemployment and historically low interest rate. I had also come across people who couldn't care less and thought politicians are all the same.