"It's just the thought of owning $1mil of a single investment class thats significantly over valued to the point that its soon about to pop taking out all my capital and leaving me owing the bank a small fortune."
All YOUR opinion and is more of a symptom of your own level of comfort and your investment strategy. Whether it is $200000, $2m, $20m or $200m everyone invests money in whatever they like to a level that they are comfortable with. Sure if you are taking a $1m shorter term speculative punt then it may worry you, but if you are taking a longer term view then any negative impacts (assuming they occur) are of less relevance.