Well, we have been through several versions of the statistical significance debate on this forum, so no surprise to see it come up again.
I would say this:
Pablito is correct that a study with a p value of 0.05 (or even less) gives a greater likelihood that any positive result is genuine, ie that it is not just a result of random variation.
In a clinical trial you are attempting to disprove the "null hypothesis" that "the treatment has no effect." If the treatment appears to be having a positive effect, then a lower the p value makes it more certain that you have indeed "disproved" the null hypothesis.
The downside of this is that to get a study with a lower p value you generally need a larger sample (many more participants) and probably a longer timeframe too. Remember that the Independent Monitoring Committee recommended that IMU reduce the number of trial participants because they felt that safety had been demonstrated and there was sufficient evidence of a positive effect.
So with a smaller sample size you get a higher p value and less certainty that your positive result is valid. On the other hand, you do save a lot of time and money. Investors tend to get antsy about the time taken for this kind of research, and back when the study was designed Imugene had far less cash than they do now. So you can see the upside of accepting a lower p value, particularly when IMU always saw Her-vaxx as part of a combination therapy and knew that further studies would be required
If IMU was carrying out a registration trial - to get FDA approval for Her-vaxx - they would certainly have been aiming at a larger sample size and a study with a much lower p value (greater certainty that you have disproved the null hypothesis).
But they were not doing that. The Her-vaxx study was a Phase 1 using Her-vaxx alone (to test safety) which then morphed straight into a limited Phase 2 study using Her-vaxx in combo with the current Standard of Care Chemotherapy. The primary end points (goals) were to determine:
1. Safety
2. Optimal dose for the Phase 2 study
3. Clinical efficacy based on Overall Survival
The study was never regarded as a registration trial - the third end point of "clinical efficacy" was only ever going to be indicative.
I think the mathematical purists dislike reducing a p value to a probability, but it does make it easier to understand. With the p value of 0.1 used in the study there is a 90% chance that the result is genuine ie that Her-vaxx had a positive benefit. Of course that also means a 10% chance that is was indeed just random variation. No-one would register Her-vaxx for general use based on a study with a p value of 0.1, but from a research/investment perspective - it does give sufficient indication of success to fund more detailed trials - and that is exactly what IMU is doing.
The fact that the Her-vaxx data presentation was accepted for a speaking slot at ESMO Asia is a very good indication that the results are of scientific interest. Pablito is entirely correct in saying that a study with a p value of 0.05 would have given greater certainty.
From my perspective, as an investor, the likelihood that Her-vaxx has a useful clinical benefit has greatly increased based on the Phase 2 study data. Similar or (hopefully even better) results in the new Phase 2 studies will massively increase that likelihood.
@Pablito -
genuine apologies if I misrepresent your thoughts.Cheers
Dave
(20min delay)
