Chronic Lower Back Pain - the sleeping giant., page-31

A very nicely written and argued post dachopper and a welcome relief to the junk three liner posts that pollute these threads. In the interests of balance let me present the ying to yang.In my opinion the webcast will likely be a fizzer. The CLBP results are 5 months old now and it is unlikely anything new will be presented. In my mind it is even debatable that the presentation will be considered to be a “clinical trial results presentation”

1. What makes a “clinical trials results presentation”.

Since about 1960 clinical trials have been thought of as medical experiments where prespecified primary and secondary outcomes are tested using statistical techniques specified a priori in a statistical analysis plan.

The trial is implemented and data are collected. For each of the outcomes a statistical test is applied and a important value pops out – a p value. The results from these analyses are said to be “the clinical trial results”.

Turn now to the MSB biotech presentation released to the market 12/1/2022.

https://www.asx.com.au/asxpdf/20220112/pdf/454xw71f5hr4x0.pdf

Scroll through the CLBP results slides shown on pages 24-29. On the pain slide, shown below, a “nominal’ p value is reported.

Nominal p values arise from statistical tests that were not specified a priori in the statistical analysis plan. Take a dictionary definition of “nominally”:

“Something nominally true is true inname only. This refers to things that are only titles or formalities. It'sthe opposite of "really." Words with the root nom relate to names,and nominally refers to situations where someone has a name or title thatdoesn't carry a lot of weight.”

The statistical meaning is the same. Nominal p values are true in name only. With “real” p values produced from analyses of pre-specified registered primary and secondary outcomes as per the a priori statistical analysis plan.

Now go through the CLBP results slides - every p value in the presentation is labelled as “nominal”. Not one real p value is reported.

Out of the seven (1 primary and 6 secondary) outcomes registered MSB report the results for only one - the minimal pain responder endpoint.

The results for the primary outcome and the remaining five secondary outcomes are missing in action. With only 1 of 7 prespecified outcomes presented this is non-compliant with both ASX trial reporting guidelines and universally adopted standards for clinical trial reporting (eg CONSORT).

So ask yourself. Do you think you are seeing a “clinical trial results presentation” or are you seeing the product of a statistical exploration of a dataset that just happened to be generated via a clinical trial?

2. Why did the primary outcome of this trial fail?

I think every investor / analyst on the webcast would like to better understand what went wrong; why did the primary outcome fail.

MSB has never included the actual primary outcome results in any presentation. But those results are required to be lodged on the clinical trials registration site.

For the primary outcome (the composite of pain, function and no Post Treatment Intervention: [PTI] at both 12 and 24 months) The proportion of responders in the stems group (0.267) was lower (poorer) than placebo (.312). There was no difference (either statistically, clinically, practically) between the proportion of responders in stems+HA (.335) vs placebo (.312). In short, the primary outcome wasn’t even trying to do the right thing.

Usually, P3 failure can be traced back to P2 problems. Here we see the P3 composite primary outcome was based on positive results in the earlier P2 trial. But in the P2 trial this outcome was created post-hoc; it wasn’t pre-specified. In addition, the P2 results for this outcome were not statistically significant at both 12 and 24 months with success at both 12 and 24 months needed in the P3 trial.

In the P2 trial the numbers in each of the 4 groups were very small (between 20-30). The P2 authors acknowledge the small sample size as a limitation, which in their view limited statistical power and the interpretation of the results. Why was the P2 study under-powered? According to the authors “As appropriate for an early phase study, the planned sample size was not prospectively powered …”

So there’s your root cause for the failure - a lack of rigour in P2 - no formal powering.

With the benefit of hindsight the weaknesses in the P2 trial are not hard to spot.

The response format for a VAS pain score is typically pain on the day the questionnaire is administered. It is a subjective measure. Clinical staff were unblinded to treatment allocation.

In the P2 study in the treatment group around 40%-50% were responders. In the control group (n=20) there 3-4 responders at 12/24 months. People have good and bad days. The effect here depends on less than half dozen control group participants not having a good day. Or in the reverse; it relies on half a dozen participants in the treatment group not getting a cup of tea and a tim tam at their outcome study visit.

3. Meaningless results in the presentation.

Now at this point posters here will say - well so what. We all know the primary endpoint failed. But the stems+HA reduced pain as shown in the pain figure above; and that’s what will be proved in the next study.

I think the results in that figure are meaningless. This is because the analysis fails to account for the confounding of change scores with patients receiving post treatment interventions. The P2 trial authors explain the problem like this:

Paraphrasing. You receive the stems. A year later the treatment hasn’t helped. You receive a new treatment (a post treatment intervention) which reduces your pain score at 24 months. The improvement between baseline and 24 months is due to the new treatment and not the original treatment. Failing to control for PTI therefore renders the pain (Visual Analogue Scale: VAS) change assessment meaningless.

There was one correct way to assess the pain effect in this trial. That is from the first secondary outcome registered for the trial. That outcome is the proportion of patients in each group who had a clinically significant pain improvement and did not have a PTI.

So you have to ask yourself. Why isn’t the secondary outcome for pain that adjusts for PTI reported? Why the switcheroo to an outcome that MBS said in the P2 trial results was meaningless.

To summarise the case for the prosecution. Clinical trial result presentations report the results of registered primary and secondary outcomes analysed by techniques specified in a statistical analysis plan. This produces real p values (not nominal p values). In the presentations to date the results from only 1 from 7 outcomes have been reported. By my calculation the bull dust factor here is 85.7%.

In relation to the proposed new P3 study? The current P3 trial failed arguably because of retro-fitting a post-hoc endpoint to P2 trial data that were methodologically weak (under-powered). The proposed new P3 study is being built around retro-fitting an endpoint created post-hoc that MSB said in the P2 trial was meaningless. Doing the same thing and expecting a different result is …….?