In a nutshell Davisite your opinion is that BIT has been data dredging because they performed statistical tests for hypotheses that were not pre-specified. And this should never be done.
This is what you said last time and so I’m not sure what has been extended. But lets continue anyway.
The principles that underpin the statistical analysis of clinical trials are expressed in a real mouthful of a guidance document called the “International Conference on harmonisation of technical requirements for registration on pharmaceuticals for human use – E9 statistical principles for clinical trials”
https://www.ich.org/products/guidel...atistical-principles-for-clinical-trials.html
This guidance is very important because it has been adopted by the FDA, EMA and most everyone else. The advantage of the guideline is it helps settle arguments quickly.
An important first step – as we have discussed previously Davisite this trial was a P2 exploratory trial.
What does the guidance say about the analysis of P2 exploratory trials?
“In contrast to confirmatory trials, [exploratory trials] may not always lead to simple tests of predefined hypotheses. In addition, exploratory trials may sometimes require a more flexible approach to design so that changes can be made in response to accumulating results. Their analysis may entail data exploration. Tests of hypothesis may be carried out, but the choice of hypothesis may be data dependent.”
Let be crystal clear about what is being stated here.
The ICH is saying that in exploratory trials statistical tests might be applied that are not pre-defined, that entail data exploration and that are data dependent.
All the things you claim Davisite BIT should not have done the ICH – FDA – EMA says feel free to go right ahead.
To a lay person I’m sure the attitude of the ICH is counter-intuitive. In effect aren’t they giving a green light for data dredging which must be a bad thing.
What gives?
I think if you asked the average life sciences investor (or benchtop scientist Davisite) to explain the difference between a P3 trial and a P2 trial they might say something like P3 trials are bigger than P2 trials and regulatory bodies require P3 trials for approvals.
And if you followed-up with asking well why doesn’t the FDA approve things just on P2 trials they would say that well because the P3 trial is much larger than the P2 trial the FDA would be more confident that the results were true.
In effect they are really saying that P2 trials are just little versions of P3 trials. And from this it would follow that the approach to statistical testing would be the same.
But this is wrong.
P2 trials are performed to screen for candidates in which to take through into expensive P3 trials. How industry goes about that screening process is up to them; and hence the rather permissive attitude to statistical analysis prior to P3. No-one wants to (or thinks it would be a good idea) to put a straight jacket around how people use statistics in that screening exercise.
But this comes with a catch. If industry is not clever about assessing efficacy signals from P2 they will take things through to P3 and fail. And it is the company that will lose a lot of money not the FDA.
For BIT investors here I think my message would be there is good news and bad news.
The good news is in my opinion the inferences that BIT are doing something dodgy or scientifically inappropriate with their statistical analysis of this trial are completely unfounded.
The bad news is that the strength of those efficacy signals (because they are from post-hoc analyses and data dependent) are in all likelihood not as strong as people might like to think.
This is going to be particularly the case here because the efficacy signal is coming out of a small pharma company. And the reality is small pharma typically lacks the sophisticated processes and rigour that big pharma use to control their false discovery rate.
Now I am hoping (against all the odds) Davisite that the discussion can actually be extended. Hopefully we don’t just don’t go round and round on this post-hoc data dredging issue because you don’t understand the differences between trial phases.
A useful line to explore might be how on earth does the poor old biotech investor assess the strength of clinical trial results coming out of micro-cap P2 trials if what I have said is true.
One thing I would like you to do though is to request a moderator to change the title of this thread to something that is not inflammatory to BIT. A neutral title – say – “Statistical issues in the analysis of BITs P2 trial.”
(20min delay)