Interim Analysis Debacle

“Interim trial data have only rarely beenmade accessible to current or potential trial participants, on the basis thattrends in accumulating data may lead current participants to withdraw from atrial and dissuade others from joining. The standard [most accepted] policyused in the large majority of trials is to keep accumulating data confidentialto the DMC and the trial statistician who prepares reports, although thispractice has been questioned.” Quote from Grant et al., 2005 ¹

The reason it has been questioned is in thearea of ethics and patient consent. Is it ethical to withhold details of howthe trial is going (i.e. the efficacy results of an interim analysis), unlessthere are safety issues? From Edwards et al., 1998²

Regardless, the results of an interimanalysis will have one of the following effects:

There is no or a slight significantdifference in primary outcome between active and placebo; major safety problems– trial may be terminated due to harm. Unknown if treatment is useful

There is no or a slight significant difference in primary outcome between active and placebo; no safety problems – trial will continue.

There is a strong significant differencebetween in primary outcome between active and placebo; no safety problems:

– P>0.001 – trial will continue.
– P<0.001– (Peto et al.,1976 ³) trial will be considered successful and terminated.
– P<0.0001(O’Brien and Fleming criteria⁴) trial will be considered successful and terminated.

So, unless the interim analysis is significant at P<0.001, the trial will continue. Only the DMC and the trial statistician will know if there is any sign of efficacy by significance in the primary outcome at a less significant level.

“Hereare four possible alternative plans for two-sided testing with overall o~= 0.05. Plan A has fixed nominal levels each at p < 0.018. Plan B (the O'Brien and Fleming rule) has an increasing gradient of p values such that the first interim analysis requires overwhelming evidence (p< 0.0001) before stopping the trial while the final analysis (if reached) has a p value not much smaller than 0.05. Plan C is intermediate between plans A and B, and actually minimizes the average sample number for that alter- native hypothesis thatcan be detected with power f3 = 0.5. Plan D (introduced by Peto and colleagues) simply expresses extreme caution (p < 0.001) for stopping at any interim analysis, so that the final analysis can take place virtually without adjustment for previous looks at the data. AlthoughPlan A has the greatest savings in average sample size when alternative hypotheses can be detected with high power, it also requires a larger maximum sample size than the other plans. Because most clinical trials are struggling to achieve sufficient power,it is usually wise to opt for a gradient of p values, though it is debatable whether this should be as extreme as the O'Brien and Fleming rule. It should be recognized that plans such as in Table 1 are arbitrary, and the choice between them does carrysome implicit prior belief about plausible magnitude of treatment difference, that is, plan A, or even a continuous sequential plan, may be preferable if large differences are expected whereas plan D has merit if smaller differences are anticipated.” From Pocock and Hughes, 1989 ⁵

“The FDA draftguidance points out that an independent DMC “promotes objectivity that benefitsnot only the participants and the trial but the sponsor as well, in thatthe credibility of the trial’s conclusions is enhanced”. In particular,remaining blinded to interim results “protects the sponsor (and thus the trial)from pressure towards premature disclosure of results due to SEC [theSecurities and Exchange Commission] requirements, fiduciary responsibility, orother business considerations” (‘SEC’ refers to obligation to releaseinformation to investment agencies).” From Edwards et al., 1998 ²

“Risks of exposureof interim data

Trials withpotential regulatory impact provoke particular attention to keeping the sponsorfrom knowledge of the interim results. The following issues arise.

● Unblinding to the sponsor could provoke further unblinding, which can influence decisions and can introduce biases says the role of any sponsor representation on the DMC should be clearly defined and control of information addressed.

● Sponsor access to interim data for planning purposes has been explicitly considered and strongly discouraged, but if necessary could be carried out under tightly controlled circumstances, for example, if production would need to be scaled up to make a drug available if a trial stops early (see question 13).

● The FDA should be notified if the sponsor is going to have access to interim data.38

● An independent statistician is recommended to act as the analysis statistician and prepare reports to the DMC.” From Edwards et al., 1998²

“The statisticalimplementation and execution of group sequential methods, commonly used toperform interim analyses (IA), are well established. These allowfor repeated analyses of accumulating data to be performed whilst protectingagainst erroneous conclusions. Despite this, group sequential theory andsimulation studies indicate that conventional estimators of a treatment effectfrom a positive IA will tend to give exaggerated results, i.e., the observedresult from the positive IA will tend to be more extreme than its true value.”6

Challenges ofestimating treatment effects after a positive interim analysis

“a beneficialtreatment effect reported at the first interim analysis (IA) may diminish at asubsequent analysis (SA). We examined three challenges in interpretingtreatment effects from randomized clinical trials (RCTs) after the firstpositive IA: overestimation bias; non-proportional hazards; and heterogeneityin recruitment.”

“In summary, the results of interim analyses should be reported carefully and interpreted conservatively because they are prone to overestimation bias and problems resulting from non-proportional hazards, recruitment heterogeneity, and heterogeneity of other treatments.” 6

ParadigmPhase 3 PARA_OA_012 trial

PrimaryObjective: Evaluate the effectiveness of PPS inreducing knee pain.

•PrimaryEndpoint: Change from Baseline (CFB)at Day 112 in knee pain as assessed by theweekly ADP (average daily pain) score on the numerical rating scale (NRS)11-point (0–10) scale.

InterimAnalysis Criteria

Theinterim analysis (IA) will be conducted by the independent DMC. Statisticalrules for the analysis will be documented in the statistical analysis plan forthe interim analysis. An early conclusion of efficacy may be established if arigorous effect size is demonstrated. A conclusion of futility may beestablished for minimum treatment effect with low probability of success.Should early efficacy or futility not be established at the IA, the trial willproceed to the final analysis with the full sample size (N=466).

PR said this 21May 2025:

“So that means the Phase 3 isup and running. We are working with a CRO, we have very ambitious timelines,and we expect that over the next 12 months we will be reporting 100% of thesubjects, that’s 466 subjects, recruited into the trial, and we also hope thatin close to the next 12 months, maybe 14 months, we’ll be having an interimanalysis reading out the results of the first 233 subjects that have reachedDay 112 followup. ”

Except, only the data management committee and an independent statistician are supposed to have any knowledge of the interim analysis data and results. If they say “the trial can stop” they either have a rigorous effect size (P<0.0001) between active vs placebo NRS pain scores for the first 233 patients that includes a minimal clinically significant difference, or they say that there is absolutely no difference and to continue is futile (unlikely with such low patient numbers).

Ifthey say “the trial can continue”, there is no way of either the investigatorsor the trial sponsor knowing what the effect size from the interim analysisactually is. The DMC is not allowed to reveal any results or data from theinterim analysis especially to a sponsor (Paradigm). They cannot makeany announcements other than that the trial will continue. The fact that theinterim analysis was done introduces overestimation bias and problemsresulting from non-proportional hazards and recruitment heterogeneity.

References

1. Grant AM, Altman DG, Babiker AB, et al. Issues in data monitoring and interim analysis of trials. Health Technol Assess 2005; 1–238, iii–iv.
2. Edwards SJ, Lilford RJ, Braunholtz DA, Jackson JC, Hewison J,Thornton J. Ethical issues in the design and conduct of randomised controlledtrials. Health Technol Assess 1998; i–vi, 1–132.
3. Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer 1976; 585–612.
4. O'Brien PC, Fleming TR. A multiple testing procedure forclinical trials. Biometrics 1979:549–56.
5. Pocock SJ, Hughes MD. Practical problems in interim analyses, with particular regard to estimation. Control Clin Trials 1989; (4 Suppl): 209s–21s
6. Soon YY, Marschner IC, Schou M, et al. Challenges ofestimating treatment effects after a positive interim analysis. Eur J Cancer 2024;