Math problem with results, page-197

I have an answer from Tony at Resapp.  While I didn't get explicit permission from Tony to paste the whole thing verbatim, it contains the following statements:

1) All of the participants were re-evaluated to be aligned with the design of the US study, including the clinical case definitions.

2) The study clinicians went back through all patient’s medical records and re-diagnosed all the patients as per the US clinical case definitions.

3) All of the participants were diagnosed with exactly the same algorithm, designed to match the endpoints designed for the US study.

So if you accept these statements it's reasonable to conclude that (using 95% confidence level and a population of 500 million to calculate the confidence interval):

a) If the symptoms and coughs were captured in the same conditions as they were in Australia then the US trial would produce similar numbers.

b) Except for pneumonia the Australian results were substantially higher than the 75% at 95% confidence.  The smallest sample was 53 patients and 90% overall agreement which produces a confidence interval of just over 8.  So you can be 95% confident that the US trial would produce a result between 81% and 99%.  The others are all tighter due to higher agreements and/or larger samples.

c) Pneumonia's result of 81% agreement for 123 patients at a confidence level of 95% produces a confidence interval of just under 7.  So the US trial would be expected with 95% confidence to produce a result of between 74% and 88%.  If you only take the lower cut-off then you can be 97.5% confident that the overall agreement in the US trial would be over 74%.


From this I conclude that we should put to bed any concerns that the management went into the US trials with foreknowledge that the trials were doomed to failure.  In fact based only on the results and confidence intervals you'd be justified in arguing that any effort put into a plan B would be a waste of time. As long as the Australian/Indonesian conditions were replicated.

The question still remains - and I have no specific standpoint here - as to whether the management's expectations of replicating the Australian or Indonesian conditions were realistic.  Perhaps they weren't.  Perhaps the management's risk analysis and response planning was inadequate.  Perhaps the product won't ever be usable in a real-world clinical environment.

I don't know.  I'm quite interested in the answers to those questions.