As Daimian Drum said to the jurno, there where rumours about the affair but there was no suggestion that they where in a relationship. Also when does a casual relationship turn into a serious relationships, dare say it just started as a fling and probably didn't get serious for some time. Of course once she became pregnant Barnaby had some serious decisions to make