Just wanted to clarify: I make good money and returns, but not tons and tons as yet. And I am not a millionaire as yet, however, I am well on my way..
I totally get what you are saying because I have been waiting for a pull back for ages too however, whilst making profits on the way up...
I guess as I am writing this post you are really hurting seeing the market come down so much. But it is ok, you are on your way to fix it all and to make money on those moves in future.
The greatest challenge I see traders have is that they don't know 'what makes them tick' and that they lack methodology and structure to implement strategies that help them to counteract any habitual and most often natural behaviour that is counterproductive in the context of trading.
So they try to fix the symptoms such as lack of discipline, fear of pulling the trigger desperately searching for ways to become confident, stop being greedy and still wallowing in regret for the profits they haven't taken and the losses they 'should' have cut earlier.
Let me give you specific examples based on your self-analysis above. Of course my response is still very general and I hope it will help you continue this journey:
1. '"convinced" that we would get a pullback'. Oh boy, how often did I do that and how often did I pay dearly for it. So, the symptom here could be a form of 'rightousness', the need to be right. However, what could be the root cause? We human beings need and look for structure and certainty. Feeling convinced gives us certainty and peace of mind for example. (It could be a totally different reason for you personally, I just want show you the structure of my assessment and you can maybe do some self-assessment). So, now we know what the core problem is, we want to feel certain. What is the solution? So, this is what I do: I create a gameplan like a martial artist. You know, when they prepare for a fight, they have pretty much the same conditions like us traders. When they are in the ring every move counts. So the good ones do shadow boxing. My opponent can only do front kick, back kick and roundhouse kick. If he does a back kick what will I do then? Take this principle into trading, the market can only go up, down or sideways. So, when I do my weekly and daily preparation I don't say: my analysis says the market goes up or down, I say: ok, if the market goes up, what will I do? If the market goes down, what will I do? Once I am in a position, what will I do then? This way I become pro-active, it takes away my need to be right and all the other survival instincts. I am well prepared for any movement in the market because I have practiced what I will do in advance and that gives me certainty and self-confidence.
2) Now to your analysis. I say that with all my love and experience and coming from a place of wanting to be constructive ok? Now, here is the thing: Based on how you describe your trading decisions in your post, it seems to me that you have to work on your methodology and systematic approach to trading. if your trading system has kept you in the shorts or has not given you long bias, you do need to do some work on your trading system dudes... If your trading system has not given you clear exit or cover signals, you do need to work on your trading system. It sounds to me that you trade too much based on your opinion and 'mindset' instead of hard and fast trading rules and strategies.
3) Judging by the choice of your words I have the impression that at the moment you feel a bit at the mercy of the market, 'positions gap up on me', 'should have' etc. I always like to say, don't say I should have could have because if you could have should have you would have, know what I mean? :-) Say, 'next time I will', 'next time I do'... that will put you back in the position of power...
3) 80% of trades where profitable at some point. It seems to me that you need to work on your exit strategy. For example what I do is as soon as a position moves into a profit, I put my stop loss even if it is 1 point in profit only. Ok, some of the time it takes me out just to continue merrily on it's way, well, then I just wait for the next opportunity. But I can assure you that the majority of the time it has saved my butt big time.
4) you say today was the height of your stupidity. I believe that it was rather lacking some sound trading methodology. Because there are a few trading systems out there that I know off that would have gotten you in long at 3935 and out at 4020. So, it is not stupidity, it seems to me simply lack of skill.
Let me tell you a story: I once went with my boyfriend on a race course, he loves racing cars. As we approached the curve with a big concrete wall, the brakes faild and I covered my eyes and cried "Hooonnneeeeeyyyyy, sh#$@#$@#$". But my boyfriend just laughed at me and said it's ok (we drove 180km... and he says it's ok as we approach the concrete wall). He did something with the gears and the handbreak and the car slowed down stopping safely on the green strip. NOw if that had been me driving, I would have hit the concrete wall, not because I were stupid, but because I wasn't trained for such situations, whereas my boyfriend is. Or once we drove from Sydney to Melbourne and it was pitchblack and the lights went off. Again I panicked but he simply swithced on the highbeams, because he knew that they are differently wired. We stopped at the site of the road safely and got the lights fixed.
So, what am I trying to say? It is all a matter of skill and knowledge, not stupidity, so be kind to yourself ok?
5) I agree with you on the subject of luck, we all need a little luck. However, luck once again is often closely related to experience, and experience is the greatest yet the most expensive teacher. So, just see it as having paid for university and soon you will have your PHD Dr. Caylus :-).
So, last but not least here is what makes me successful:
1. I have a clear step by step written down trading system that leaves no room for interpretation or bias or mindset or opinion. I have a big ego and that likes to get into the way of my profits, therefore I have found a methodology to manage it. I have created my own system based on successful traders who I mentored with. Not sure if I am allowed to reference anyone here, so I won't.
2. As my daily and weekly trading preparation I consistenly study a few successful traders but more with the focus of wanting to learn more on the side of technical and fundamental analysis, broadening my knowledge by learning from other trader's experience.
3. I also have a strict routine for my trading preparation every beginning of the month, every beginning of the week and every day before the trading day starts. This way I make sure I am well informed and prepared, feel good about myself, balanced and confident.
4. I have a very holistic approach to my trading, my head, my heart, my health and my happiness all need to be looked after and the money will come with it. Therefore I eat healthy, I continously work on my personal development, every day I do things that make me happy like going for my run on the beach, yoga, catching up with my friends etc.
So, you now Dr. Caylus, there is a lot more things that I could write here. I hope the above helps you a little with your self assessment.
Just keep up the good work, add some technical analysis skills to your fundamental analysis skills, keep working on your mindset.
Let me know if you have any further question, and if I can I am happy to help.