Not sure If I'm stating the obvious but for those that need some help in understanding this....the big take away for you is that all this talk of p values and drug effect sizes are going to be VERY MUCH DEPENDANT ON OUR SAMPLE SIZE!
Yes that's right, you take a smaller sample size (err..let's call it
n for number!)...where n is small...oh I dunno, how about 20 odd
+ ( + = I chose this number cos that's roughly how many patients we actually ended up having in each grouping of 008!!)...that's really quite small...imagine, on the setting off on our long clinical journey the bio statistician along with the authority and the sponsor all came up with a Phase three that involved a fair bit more than 20....in fact it was in the hundreds...ie our current Phase III.
What I'm trying to say is that 20 patients isn't a lot..and yet, in some regards and in some observables in our 008 we STILL achieved the holy grail (well a little less holy than DMOAD then) namely, Statistical significance...this essentially means we achieved under that 0.05 p value bench mark...that was truly remarkable.
So to get a p value that's quite low is good, to get it fairly close to 0.05 is also good....but it also means that in theory, if the drug effect size remains constant then we will achieve this better p value with more numbers. Others have said that, this isn't revolutionary, I'm just trying to clarify it for a number of folks (well, including me) so that we all have a clearer picture of what we have to do and where we are hopefully heading!
From the last paragraph I want you to really take away (gee there are more take away's here than some of the smaller fish and chip shops that only have seating for 6)...the following...
The smaller the p value is, the stronger the evidence is BUT we also need to take into consideration the sample size. Also we need to consider what we are testing..if we are testing how deadly our drug is, we are going to place even more emphasis on this evidence even if it is a very weak p value......but if we are testing whether there are positive benefits of our drug then we can afford to be more tolerant.
In this way I wouldn't get too hung up about the exact 0.05 figure, I would be more inclined to run the same sort of test with bigger n so that it is adequately powered to show the real effect of our drug in numbers.
Lots of take aways? Some fish and chip shops are really just take away only.Another way to think about the p value is:
"...the greater the difference between two observed values, the less likely it is that the difference is due to simple random chance, and this is reflected by a lower p-value".1The best part about the above statement is that we already have had a great look-through via the sheer consistency of our drug (think 005, SAS, EAP etc)...this is the reason the authorities also make us do a final trial in much greater numbers. The great thing for us is that we have already seen some good p values that are better or close to 0.05 and have achieved some statistical significance already despite the n being so extremely small. As long as the drug effects are consistent, the application of the protocols are all consistent...the stats can only get better.
Ok I get the theory Mozz, can you now apply this to us?
There was a stellar announcement last year in September that caught my eye and I have a feeling it didn't really catch many of the non Hot Copper PAR eyes....
The above statement was pretty remarkable, specially in light of the below statement all within our 56 day read out announcement at the end of September last year...
Statistical Significance.
What does this mean? It means that in our very very small n...they achieved a thing that they weren't at all expecting, in fact they were advised that they wouldn't get it....
This means that although the patient numbers were tiny in the sub set...the drug effect size was so great that they achieved a good p value. This will only be magnified as our n goes up.
When will our n go up? Well the first batch is happening as we speak......dosing study...yep, that's 117 per cohort in our dosing study. The numbers bump up again in our upcoming 002 part 2.
All very good, but here is a warning about the p value:
Even a low p-value is not necessarily proof of statistical significance, since there is still a possibility that the observed data are the result of chance. Only repeated experiments or studies can confirm if a relationship is statistically significant.Remember that bit I had about the observations we have already had, that's what I'm alluding to, the more consistent our data is over repeated experiments (trials/studies), the better it is for us going forward, we have already had a great taste of this not over months but over years. In fact it has been the opposite for us, not only we achieved good consistent results but the batches, at least in terms of the SAS and in our EAP programs, have achieved better results than the original batches. This bodes well for us.
Here is a perfect and real example of what I'm talking about, we see the below taken from our latest results, we see that C2C and ARGS (synovial fluid) was reduced and the p - value was low, we note the others also reduced but no p values given, I presume that these were greater than 0.05.
In theory, based on how the stats work, I'm also presuming that you increase the n, we will get better results in terms of p values.Oh wouldn't it be just so nice if we didn't have to wait for this sorta observation to play out...
We don't.
What?
We don't have to wait..
Whaddya mean?
We have it already, we have already seen it...
Yeah yeah but was it in a 'proper' setting? Double blinded well controlled placebo armed trial etc... ??
Yep!
Mate!! Read what's in that red circle above...we already got crazy good p values, and what was the n?
126! See green box below on the same slide:
Another classic example is PGIC....what did we observe when our n was hardly 17?
...and what did we get when our numbers were higher back in 005?
That's what I'm talking about! It's those kinda p values we will end up delivering (my views) in our full trial to come...
...we have the clue today...we have the clue now......it's the future read outs in numbers that will really be quite exciting.
Mozz
DYOR recommended
For those that thrive for more, here is a story.
6There was a drug where they did a massive trial. I'm not talking 3000 patients, nope, not even 7,000. I'm talking a full 22,000,
Jeepers, they were delighted when they got the read out and p value was a nutty p < .0001.
Yeah the regulators and the sponsor were jumping for joy. The problem is that this masked the drug effect size, so much so that it in turn was tiny!! What they eventually found was that a effect size was small, and the R
2 should've been observed, what were the confidence levels. The risk difference ended up being 0.77% and the R
2 ended up being only 0.001. This meant that only a tiny one tenth of 1% of the risk of the primary endpoint (in this case the incidence of myocardial infraction (heart attack) could be explained by the drug...
Note: R
2 can be thought of as how fit is the model.
Err no, not this model...how fit is the statistical model..What was the drug?
Aspirin.
They finally (after a few years) ended up changing the prescription and the advice on who should take it and when.
Mozz Thought:
Wouldn't it have been interesting if they had followed our pathway and done the study with a low n first? We are ALREADY seeing a noticeable drug effect size when our numbers are anaemic imagine when we do our main trials...we should, according to me, hit that statistical significance tipping point quite early. We simply won't need all the conservative numbers, potentially. Of course it will depend on when the stats are compiled, when the interim read happens, how the screening compares to our 008 and many other such factors.
I watched John Wick 4 the other day and my sons and I always stay back for the credit on those Marvel Films as they give you a bite sized extra scene right at the end...we did this for this movie as well, knowing they wouldn't have anything...but they did!
A bit like that, here is another snippet for those that have read right down to here.
When typing out the above I had a very feint recollection that the first time we read about PAR's statistical significance wasn't actually in Sept last year, it was a number of YEARS before that...while doing the research for this post, I stumbled across it.
On the 15th of April 2019, Paradigm's 005 Secondary Endpoint announcement mentions it not once, but three separate times on the one page...Quite a read:
've had a p value post on my mind for some time, but with this latest release of data (008 6 month in case you were off visiting Mars)...it's become fairly topical.
(20min delay)
