Ann: Letter to Shareholders, page-199

I’ll have a crack at explaining it Davisite – because I also don’t see a problem with this figure and how BIT have applied that statistical test.

From the figure CD8 mean levels do appear to increase in the first few weeks in the bit225 cohort compared to placebo. This fits with BITs theory those patients were exposed to a unique antigen source.

And at the third time point guess what these means levels were statistically different. Funnily enough this was tested using a Welsh T-test which is more conservative than a student t-test and less prone to type one error.

This tells you one of two things.

Either the analyst was very skillful at using the most appropriate t-test in this particular situation (where you had unequal sample sizes) or they had dumb luck because they were using the R package which spits out the Welch t-test by default in contrast to most other packages which default to the student t-test.

Which ever way - skill or dumb luck the test is appropriate for the data and in the context of this trial.

Your concerns about post-hoc or multiple testing are just misplaced. The trial was exploratory – generate hypotheses for confirmatory testing – and the data is suggestive of exactly what BIT thought (hoped) was going on.

The fallacy in what you are implying is that somehow due to this post-hoc multiple statistical testing nonsense this finding of initial CD8 level increase has poor positive predictive power – ie wont replicate because it isn’t true because of the approach taking to statistical analysis.

This is just wrong – what is determining the reproducibility of this finding is the basic design (the strengths / weaknesses) of the trial – not whether or not some statistical test has a p value of whatever it is.

Hopefully you can understand what I have outlined above; even though I don’t claim to be an expert about anything. So I just must be good at googling stuff. Perhaps you can clarify for people what exactly is your experience in the statistical analysis, reporting or reviewing of clinical trial data and methods.