PAR (ASX) The Probability of Success

ust last Thursday morning Paradigm gave us a webinar with a focus on stats and the potential Probability of Success.

This one is going to be an interesting one. I've bumped this up ahead of my usual Sat night specials.

Please of course, enjoy

DISCLAIMER

I need to start off this post with a disclaimer. A lot of the below is my personal interpretation and I will be skipping a lot of pure statistical detail and will be over simplifying a lot. As part of the disclaimer, in attempting to making things simple, it also may give you (and me!) a somewhat false impression that I'm covering everything, every aspect of the statistical plan and trial design.

This isn't the case.

There are a LOT of moving parts in a clinical trial.

Lots of moving parts in a clinical trial, lots of moving parts in the manufacturing of iPPS !

Just because the theoretical stats say one thing, it doesn't mean there can't be anomalies, outliers and varying observed effects that we may encounter. This can change results or skew data. In no way am I saying that any data or statement I come up with WILL happen or indeed am I making any assumptions of data that we WILL see.

Speaking of skewing data, let me have this one dad joke before we prime you with some heavy stats!

Q. How many statisticians does it take to screw in a light bulb?

A. We really don’t know yet. Our entire sample was skewed to the left!

Less jokes, more stats Mozz...ok let's go!

INTRO

The world of bio stats can get complex. I thought it was admirable that PAR attempted this. Tonight I want to try and simplify it a bit more to give you a better understanding of what they said and what it might mean for us.

Paul and Co presented us with a chart to start us off with how many candidates generally fail even before they make it to a P3. Guys, we are at this Phase 3 stage now. In hopefully mere weeks, we will get the first dosing ann. and we are off! The wait, at least for me, gets easier then because I know it's totally out of my hands, it just needs to progress in the background, as every month gets ticked off in the Mozz Study by flipping my calendar, it's another month closer to the inevitable!

Bring it on! When will it be Mid 2026? Anyone?

So what was the crux of what PAR presented. What does it mean for us?

They took us through two prior Phase 2 studies and how they performed. We actually got some never before seen data.

Let's go down a level and investigate some more. I know there will be a number new to us that don't even really know what these Phase 2 studies are. Let's quickly do a super brief recap.

005 was a Phase 2b that was completed way back at the end of 2018. It studied the positive effects of iPPS mainly concentrating on observables from the serum (think of that basically as the bloods).

It had a total number of patients of 121 divided up into Placebo and Active.

The beaut thing about this study was that it was quite comprehensive and broad ranged. It not only looked at Womac pain but it reported back to us on some other great measures, ADL, Function improvement but even more measures like structural, think BMELS. Not just top line but we got some incredible data on size, weight and even grade of BMEL reduction. We also started getting back information on COMP measures and ADAMTS 5 levels, actual OA biomarkers. How were our patients performing not just on the level of pain, but a number of great measures, like the biomarkers. Biomarkers of a diagnostic nature but also of a prognostic (predictive) nature.

008 was a much smaller trial but a much more recent trial, also a Phase 2b but it had the aim of connecting what was happening in the serum (005) and what was occurring in the synovium (inner capsule within a lot of joints like the knee for example).

Now there was a really big difference between these two trials in terms of drug effect size.

To be an eligible patient under 005, you needed to have BMELs. These are lesions that occur, not every OA patient has them, but a lot do and can be physically of all sorts of sizes. However, 008 I think of the inclusion criteria to be more like general POP (population), you could, be admitted into this study without BMELs.

There were also a number of other differences inherent between these two trials and certainly one of these differences was the implementation, by PAR, of the learnings they had in terms of better handling the Placebo effect. There would have been also a number of other differences between the two trials.

At the end of the day though what is our Primary Endpoint? This is really our single most important measure. The others, our secondaries, are also important but it is our primary that we need to beat.

In the Webinar we had on Thursday, PAR focused on the pain endpoint, our primary. Their aim was to give us:

WHAT this history was in terms of prior data
HOW they came up with the P3 plan but they went one further for you and I.....WHERE this could go...

QUESTION/ANSWER

Now I'm not going to get bogged down and re-hash all that Donna stated, I'm going to focus more on one particular question I asked and my major take away after I got that answer and what I did with it....

But first, we need some background....

Between 005 and 008 we got quite a difference in Drug Effect Size.

It wasn't just a small difference it was a large difference in what we call a drug effect.

The Drug Effect Size is simply defined as how well the drug works that's being tested against some other drug or against placebo.²

The higher the figure, the better it is. Of course that's subject to side effects and other interactions. Some drugs have great efficacy at higher levels but that can come with higher chances of unwanted effects. In our case we have a great safety profile, I'd like to say immaculate...but let's not get carried away, yet.

So what did we get, well you can see it in the slides ...let's take a peek:

We are focusing only on Day 112 as that is the day of our interim readout, slated to be sometime in June or July next year.

The red circle is 005 result, the green is 008.
The pooled result of those two studies was determined to be 0.30

It's prob another whole separate post on the reasons why they observed such a difference between the two studies, at the highest level it's because their aims were different of course but at a lower level, one example I can quickly and easily give you is a major difference in BMEL presence, in fact 005 required it, 008 did not, here is an excerpt from 005's inclusions ³:

There was no such clause in 008, there is also no such clause in 012.

So what was my question in the webinar?

Well I could see that we were pooling those two different results...They took 0.2 and blended it with 0.46 and got 0.3

Nice.

Then (this sounds little like instructions to a recipe)...they took the 0.3 and did a whole heap of statistical calcs with the aid of one of the best Biostatisticians on the planet for Pain clinical trials, Mr J Bolognese. Together they came up with what n should be.

Essentially, I am stating to you that the n = 466 (split into half active and half placebo) was powered based on this 0.3

In other words, in english words...in Mozz Speak ®...(now we are really dumbing it down)... if we assume our drug effect size is 0.3, then we need 466 patients over two arms to show that we are going to make it. "Make it" means getting a value of under 0.05. I will chat more about this in Appendix A for those that want to know more about what exactly a p value is. But think of that as the pass or fail...you achieve something lower than 0.05 and you win. Higher, well it's not that you may lose, but you start to say that maybe your drug isnt working as well as it could...we call that efficacy against placebo.

Err ...Mozz....your question?

Oh yes...so my question was that HOW exactly PAR, are you pooling it? I was assuming that they were utilising maybe the BMELs or maybe something scientifically, even medically to do with the inclusion exclusion criteria...and skewing it more towards the results of 005. I thought the were basing the pooling on something more specific, something involving the protocols of each study...?

Nope

Much to my excitement Donna said nope, they simply used n.

What does that mean?

That means that 005 has circa 60 patients008 had 20 or so.
So they attributed the weighting according to only patient numbers in each study.

So Mozz, why is that at all exciting??

It's great for us because it means that in reality, sure, PAR are powering their trial based on a lower Drug Effect size which is governed only by the numbers of patients in each study. In actual fact PAR's theoretical result (see disclaimer at start of this post) will be a lot higher and a lot closer to 008's result (as opposed to the 'pooled' result) because they will structure the inclusion/exclusion criteria more like 008, they would take all their learnings over all of their studies to date and will implement them in a more 008-like fashion and as the poster @boslo also mentioned, ADP is potentially more aligned in achieving slightly better results.

PAR also let us in on some additional clues, patients will be given a device which easily and consistently prompts them daily to input their latest result. So the big bonus is that this is daily, the patient doesn't have to remember 7 days ago what they felt like! It's daily. That's ADP - Average Daily Pain. It's a daily measure of pain and it isn't like Womac that goes thru a number of questions, it is ONE question, rate your pain on the scale of 0 to 10. Simple.

(As a backup [Secondary] we still have Womac as a secondary though). As I have stated in the past, ADP can result in smoother data and more accurate data, that can only be a good thing for us as we know iPPS works!

"Pain is assessed via daily scores recorded through a smart device and averaged weekly to minimise day-to-day variability and ensure a more robust signal of clinical benefit".⁰

Another clue here is that this electronic device sends out an alert that is time sensitive. I can do a repost but somewhere back in the Mozz archives there was a post I did on the nature of IPPS and how it varies depending on the time you have it! It has to do with cortisol levels. Anyway, this device is going to be good for you and I. It should result in a better data set for us.

"Automated daily alerts at a standardised time of day promote high quality data capture and time-aligned pain scoring".⁰

Another clue for us here is that while I reckon PAR are a top company in organising the trials and clinical work, it is the FDA that are also assisting us.Don't forget, it's not PAR -v- FDA.

The FDA are on our side, they too will get some accolades if they bring a successful and safe drug to a market crying out and devoid of a current solution in terms of the rampant increase in the OA disease and all it's disabilities that it inflicts upon millions of patients. Not only could they be responsible for getting a great drug to market, they will also reverse the course of the destruction of opioids if there is a viable, safe alternative that they have approved.

FDA more aligned than we think?

.
See this quote:

"FDA recommended endpoint to ”reduce recall bias and capture day to day variability”.⁰

The next question one could ask is why don't they go gung ho and weigh it upwards, ie more towards the 008 result, finally that was a much more later study, that was a study that incorporated more of the placebo controlling features that PAR learnt about over time (eg the SAS program)...and so that was a much better/realistic effect size (heck it was MORE than DOUBLE 005's effect size!!!)...why wouldn't they just go for that?

I suspect because they want to be conservative in the powering of the new P3 trial (012), they want to build in fat...they want to account for dropouts that might occur, they don't want to risk that they may just miss out.

This point I suspect is NOT well understood by many investors. This point may come out later on at, or after interim.

PROOF OF CONCEPT

Mozz, all good in theory mate, can you back it up with raw computational stats?

Well I believe I can.

We have all the inputs.

Think of it like Year 7 algebra (man, I miss those days of algebra, that was my ultimate fave subject hands down).

Think of it like x = A+ y + z

Oh those days of algebra!

(Is this sending shivers down anyone elses spine? Anyone? Goose bumps?)

We know x, we know y, we can estimate z...so we can get A by moving things around...Let's have a stab!

We have a presumed Drug Effect Size of 0.3 that's pooled. Let's check out PAR's stated 0.39 to find out what p value it would equate to.

So let's see what we haven= 117 (that's Active at Day 112, 50% of the total active cohort).
Drug Effect Size is 0.39

Now stick with me, it's a stats fest, I had Mr AI help me...but don't get bogged down in the stats, skip the formulas if you have to and see what the p value spits out to be...

The formula is:

t = d \times \sqrt{\frac{n_{active} \times n_{placebo}}{n_{active} + n_{placebo}}}

t = 0.39 \times \sqrt{\frac{117 \times 117}{117 + 117}}

t works out to be roughly 2.983

Then we calculate the degrees of freedom⁴:

The degrees of freedom for a two-sample t-test is:
$df = n_{a c t i v e} + n_{pl a ce b o} - 2$
df=117+117−2

df = 232

Finally we calc the p value:

Using a t-distribution with $df = 232$ and a t-statistic of $t a pp ro x 2.983$ :
The one-tailed p-value is approximately 0.0016.

That's basically what PAR came up with, P value of 0.002 at a drug effect size of 0.39

Now I'm Mozz, and I want to be conservative...I want to UPOD....Under Promise, Over Deliver.So no way known am I going to take the upper end, 0.46 (See that Green circle back up there in this post), I'm going to weigh it down some...I'm going to make it only 0.41

If I do that and use 0.41 as a plausible drug effect size at Interim, the same formula above gives me....

Wait for it...

p = 0.00097

Even in a two tailed scenario (this is where we look for a negative effect of the drug as well as a positive)...it still computes out to be 0.0019

EXCITING?

How is a value like that exciting? What does that actually mean for us??

Because there is a chance that we could achieve something never before seen ...an OA drug that achieves such great efficacy at P3 level that it becomes unethical to continue. In other word, as Donna and Paul suggested, early conclusion.

This means we could have a chance of starting revenue much sooner than we all think.

How does this translate to prob of success?

Well PAR alluded to it again in the presso:

So let's interrogate that above slide some more... I've added a red line to indicate what the prob of success is...its around the 98% level (see red arrow). IF, (and maybe it is a big if?) we score 0.41 as a drug effect size...

Underlined in red, we also see that PAR is saying even at a 0.39 we could be looking good.

To really use Mozz Speak ® :

Early Conclusion of efficacy - refers to the scheduled interim analysis where a trial could be stopped early if the drug shows supremely positive results.

An observed effect size of 0.39 equates to a p value of less than 0.002. This is the trigger for an early conclusion, an early halt to the trial itself, the trial would be considered a success at this point,

O'Brien-Flemming (ok we get into the stats again here, brace brace)...type 1 error is a statistical method used to control a type 1 error, that's a false positive. So in other words it looks like the treatment is successful when it's actually not.

That final point is to control the overall type 1 error at 0.025 1-sided, all that means is that there are two types of tests here one sided and two sided. Normally a two sided considers the drugs positive effect and also its negative effect, ie two sides, that's when you need to beat the p value of 0.05, if you are only looking at one side, ie the positive nature/side, the n you half that, ie 0.025, which equates to everyday language of 2.5%...so there is a 2.5% chance we managed to get the results of iPPS working totally randomly, not because the drug actually works.

WHAT THIS MEANS IN LIGHT OF OUR CALCULATIONS:

In our previous calculation for a drug effect size of 0.39, we found that the one-tailed p-value was approximately 0.0016.
Comparing this result to the criteria in the statement:
The observed effect size is 0.39.
The calculated p-value (0.0016) is less than the required threshold of 0.002.
Therefore, this means that if the Phase 3 interim analysis yielded an effect size of 0.39, it would have met the pre-specified criteria for an early conclusion of efficacy. The trial would have been considered a success at the interim stage.

Don't forget, for us to be successful at the very end of the trial (day 404 from last patient dosed), we need a p value of 0.05 or less. At interim the hurdle is harder ...it's 0.002 ! That's a toughie. (don't forget mathematically, it is HARDER for us to achieve 0.002 compared to 0.05). But the stats from the prior studies show we should be able to achieve this. PAR seem to have gone really conservative when powering the trial (ie. determining what n should be) and that's a great thing. It looks like PAR have powered their Phase 3 based on the pooled 0.3 drug effect size.

In actuality, I'm happy (wanting) to use 0.39 or even 0.41...I would never use the 0.46 but, mate, that's what they achieved in 008 ! Replicate that (or something close to that, HINT 0.39....0.41...etc) and I'm telling you, the future is not only going to look bright for us, it's going to look bright and EARLY for us. Again I state it, we are early to this party. I hope a lot of you are getting the gist of what I'm saying. There are many, many investors that are still learning about us and will pay a shite load more once they know what we know...statistically, medically...but also evidence of patients and Hot Copper Posters that have tried the drug too and have given us their account/evidence.

In concluding, we get a result anywhere close to 008 and the trial gets halted as it is unethical to continue, to be frank, and to be Mozz, you won't really care if at that point we still need to do a Confirmatory.......if we still are at 40 cents...if PAR will have enough funding...

It will all be quite academic.

- Mozz

Remember, DYOR, none of the above is advice. It's my personal opinion based on some sorta stats. Realise there is a difference between Probability of Success and making it in terms of a Commercial Success. Bio Pharmas are inherently risky and are quite a long play. The risk does not end with us starting a Phase III.

I'm not a statistician let alone a Biostatistician.

Oh and what was the usual disclaimer?

DYOR

APPENDIX A

What is p value anyway?

In clinical trials, the p-value represents the probability of observing the results obtained if the null hypothesis were actually true. In simpler terms, it helps determine if the observed differences between treatment groups are likely due to the treatment itself or simply due to random chance.

Usually for a Phase three, at the conclusion of the full trial, we need to get a p value of LOWER than 0.05.

This means there should be lower than 5% chance that we were successful because of pure luck, not because the drug actually works..

BUT, at interim, this p value hurdle is a LOT less. Why? Because you are attempting an earlier conclusion. Hey, to be let out of class early, you gotta be a good student right?

REFERENCES

MAIN REFERENCE:

0] https://app.sharelinktechnologies.com/announcement/asx/63175a83d173a5ee4771f96e37400d36

OTHER REFERENCES

1] https://about.illinoisstate.edu/gcramsey/first-internet-gallery-of-statistics-jokes/
2] https://pmc.ncbi.nlm.nih.gov/articles/PMC2791668/#:~:text=An%20effect%20size%20is%20a,study%2C%20namely%20differences%20in%20variability.
3] https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=373400&isReview=true
4] https://www.investopedia.com/terms/d/degrees-of-freedom.asp#:~:text=Understanding%20Degrees%20of%20Freedom,then%20meet%20the%20set%20requirement.

POST SCRIPT

Still confused? Do you have burning question you want resolved in regards to this post?

Reply and ask. Don't be shy. If i can't answer you, I will try and find someone that can!