They weren't just POMs. There were European convicts too, the French even sent a ship to make sure their own crims were offloaded from the fleet.