I think we are in agreement that BIT have been engaging in
post hoc data analysis, the question is if this is legitimate. I do agree that there is nothing inherently wrong with exploring any data to generate new hypothesis that can be tested in later confirmatory trials, but such
post hoc analysis needs to be clearly indicated and not used as evidence supporting these new hypothesis. Quoting from the
E9 Statistical Principles of Clinical Trials (Section 2.1.3 & 5.7) [1].
Any individual trial may have both confirmatory and exploratory aspects. For example, in most confirmatory trials the data are also subjected to exploratory analyses which serve as a basis for explaining or supporting their findings and for suggesting further hypotheses for later research. The protocol should make a clear distinction between the aspects of a trial which will be used for confirmatory proof and the aspects which will provide data for exploratory analysis.
In most cases, however, subgroup or interaction analyses are exploratory and should be clearly identified as such; they should explore the uniformity of any treatment effects found overall. In general, such analyses should proceed first through the addition of interaction terms to the statistical model in question, complemented by additional exploratory analysis within relevant subgroups of subjects, or within strata defined by the covariates. When exploratory, these analyses should be interpreted cautiously; any conclusion of treatment efficacy (or lack thereof) or safety based solely on exploratory subgroup analyses are unlikely to be accepted.
Let’s look at what BIT claimed for the CD4 & CD8 data on the Poster Presentation letter [2].
In addition, the results showed statistically significant differences in the profiles of two specific T-cell populations, namely CD8+ and activated CD4+ T-cells, during the BIT225 treatment period. These reflect the significant differences in immunological responses in patients receiving 200 mg BIT225 with current antiretroviral drugs (ART) compared to placebo with ART.
Would you agree that this statement does not make clear these analyses are exploratory? Would you consider that these exploratory analyses have been interpreted cautiously?
Phase 2 verses Phase 3
I am not sure why you are making an issue about Phase 2 verse Phase 3. Nowhere have I suggested that a P2 trial is a little P3 trial, just that
post hoc analysis does not provides statistical support for any hypothesis generated by the analysis. This applies equally to Phase 2 and Phase 3 trials.
I am not inferring that BIT are doing something “dodgy” or scientifically inappropriate with their statistical analysis, what I am suggesting is that they are not making clear in their communication what analysis is exploratory and what is confirmatory. It is hardly surprising given this lack of clarity that many shareholders are confused as to what the trial data showed and what hypothesis have strong and weak support.
Efficiency Signal
You raise an excellent point in regards the efficiency signal. What any investor in a biotech company wants to know after a Phase 2 trial is how likely is it that a Phase 3 trial will occur and succeed? I look at the data of the latest BIT trial and I see strong evidence that BIT225 does not affect HIV replication (as assessed by viral load) and weak evidence that it is suppressing sCD163 levels. BIT225 may also have an effect on CD4 & CD8 activation, but good evidence is lacking.
Given that the clinical significance of sCD163 changes in the absence of additional HIV replication suppression is unknown, support for continuing to an expensive phase 3 trial is minimal.
While this would be a judgment call by management, I think the case for even an additional phase 2b trial to answer the question of the clinical importance of the sCD163 effect is marginal. This would require a long, complex and very expensive trial as it would need to be able to demonstrate a positive clinical outcome in patients of reducing sCD163 levels using BIT225. I am not even sure it would be possible to design such a trial, but this is an open question.
What we are left with a trial that failed its primary aim and provided no robust data that BIT225 is having any clinically important effect on patients health. Does this warrant a fully diluted EV in excess of $60 million? In my opinion no, but that is for each investor to decide.
- https://www.ich.org/fileadmin/Publi...Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf
- https://hotcopper.com.au/threads/ann-presentation-of-bit225-009-hiv-1-phase-2-data.4554081/
(20min delay)