residential property investment - immoral?

Given the extent to which property investors/speculators have driven up the price of residential property, lots of Australians have been denied the opportunity to buy an affordable home. This also applies to rich city folk who buy holiday homes and price out the locals.

So is residential property investment immoral?

I will disclose that I own my own home, but that's it. And I think the answer to the question is yes.