The first objection to foreign farm workers happened during the Convict era when reformed convicts in NSW objected to further shipments of Convicts from the Irish Potatoe Famine (1845-1850) because they would have a downward pressure on farm labourer pay rates.
Shiploads of convicts had to be diverted to Qld which started the Convict settlements there.
Keep in mind that this was just before the Victoria Gold Rush which sucked in 10s of thousands of "diggers" from all over the world.
Our rapid population growth then rivalled California and the combination of Wool & Gold made Aus the richest country in the world GDP per Capita and the highest labourers pay per hour in the world with the subsequent Eurika Stockade leading to the 8 hour working day.
