HA Successful - Control Arm??, page-6

A statistical problem for generalising Phase II results to Phase III can be seen in the following (very rough) example, if we imagine that all statistically significant Phase IIs go on to Phase III, and no intervention that fails Phase II goes on to Phase III.

Imagine that the actual rate of interventions that work is 1 in 40. This would lead to a situation where (on average) 2 in 40 Phase II results would give positive results (the one with a real difference, and one by chance due to alpha=0.025 in each tail [2 x 0.025=0.05]). When you do a Phase III, your chance of success would therefore be 50%.

If the proportion of interventions that work is higher than 1 in 40 your chance of success increases in Phase III (e.g., if proportion that work is 1 in 10, then you expect 5 in 40 Phase II studies to show significant benefit, and 4 in 5 or 80% of Phase III to be successful). If the proportion of interventions that work is 1 in 80, then your chance of success would be 33% in Phase III.

The actual risk for HA being a chance result was higher than 1 in 40 because the PFS outcome was not the single primary endpoint (it was one of many secondary endpoints, any one of which could have been positive; some of these secondary outcomes would have been highly correlated, so hard to know exactly how much higher than 1 in 40).

Finally, there is another complication here, in that the Phase II and the Phase III are different drug combinations.