noob question

I just have a rather naive question about gold mining stocks...

I've owned NST and a few other gold mining stocks over the past couple of years, and I've noticed that when the stock market it going up, the gold stocks go down. I see many people give the reason for this as being "because the stock market is going up, people move away from gold therefore gold stocks go down".

Now, when the market goes down, as it did last night in the U.S, gold stocks also go down in price, and people give the reasoning that "because the market went down, gold stocks go down also".

I'm confused and clearly need some help on this. What am I missing?
If the market goes up or down, gold stocks go down either way.

I must be missing something. Can anybody give me a clue? Coz I clearly have none. Call me Alicia Silverstone.