Well yes, but Tony was inferring his colleagues would not have been complaining if the COS was a man instead of a woman. He was inferring discrimination by his own side towards women. He was seemingly doing a Gillard and admirably throwing the misogyny card towards his own colleagues. There is an irony in that given the one Gillard threw at him.
