Well I'm Aussie and Have never used Pom in an insulting way nor heard it used as a derogatory term.
What some people dont get is that it is as benign a word as Bastard - more a sign of friendship than insult.
Maybe I'm just lucky and also all the POMS where I have been.
