Its like there is a little missing puzzle piece here. The sample size is being increased to increase the power on the efficacy outcomes. This costs money which NEU doesn’t have and so the Lang Walker ATM machine needs to be visited.
Is this a good thing or bad thing?
Well it depends what the power was before the change to what it is now. Neither of which the company is telling you.
What is power again? Power is the probability to detect an effect when it is true.
So a back of the envelope power calculation.
Which efficacy measure on to which to base power calculation on?
The Motor Behavior Assessment (MBA). Why? This is the efficacy measure that that has been agreed with the FDA for the planned pivotal P3 study.
Mean (SD) MBA scores at baseline
From the conference poster (24/6/16) the first trial reported means of around 49 (SD not shown). These are similar to a paper (Lane 2015) shown for MBA (mean 53 SD 13) for Retts children and adolescents. Use these estimates as baseline estimates.
Change to be detected.
From the first study the poster shows MBA change of around 2 points. Note this change estimate is coming out of a model (most likely a ANCOVA) and showing least square mean estimates. There is adjustment for significant (<0.01) baseline covariates which will have the effect of narrowing the standard errors. A modified ITT sample (not explained) is used. Quite a bit more complexity here than people might have gathered from the first announcement of these trial results.
But lets assume with the higher dose (200mg/kg: compared with 70 in the first study) in this potentially more responsive group (children and adolescents) in this second trial we get double the effect – 4 points. The MBA has also been revised and might be more sensitive to change. Add a point. So we seek to detect a 5 point MBA change.
Sample size
Before the sample size change 16 subjects in the 200mg/kg group versus 16 in the placebo. Following the change 26 subjects in both groups.
Statistical test to be used.
For simplicity assume a straight mean comparison between groups at outcome (a t test).
Perform calculation using free software.
https://www.stat.ubc.ca/~rollin/stats/ssize/n2.html
Placebo mean 53 SD 13. High dose drug group 5 point improvement mean 48 SD 13. Alpha (p<0.05), two tailed. Solve for power when N is known.
Results
For a sample of 16 in each group power equals 0.19. With the increase in sample size to 26 in each group power equals 0.28.
Caveats
An ANCOVA model (a not a t test) will actually be used which is more powerful. This is because it will use information from the baseline score. But this is difficult to illustrate using free software. It also requires an estimate of the correlation between MBA assessments at baseline and outcome. The inclusion of additional covariates will also increase power. Perhaps best case power from 0.28 to 0.40. .
Interpretations (musings)
Companies do not increase sample sizes because subjects are lined up at the door. This idea has zip scientific merit and would not be approved by a IRB. Rather a case has been made that the benefits of the increased power have value over the above the risks / inconvenience to subjects.
Everything in speculative biotech is positively reframed. The sample increase is presented as taking advantage of a great opportunity. The truth is the efficacy power here was very low and the sample size increase improves this .... marginally.
The exact power here has only become clear following the agreement reached with the FDA about the endpoints for the planned P3 trial. This drives the decision about the sample size for the P2b trial not subject availability. The dots are there ... but few investors will join them.
The power of a study is hardly a national secret. It is often reported in trial registrations. The fact NEU has not reported it, given increasing this is the purpose for the new funding arrangements .... is well .... interesting.
Quite a bit of mucking around through dribs and drabs of information released by NEU was required in order to perform a simple power calculation. Even then guesses at parameters are required. At a MC of under a $100m for a biotech the reality is you are unlikely to find good publications of trial results or protocol papers where everything is set out for you neatly and transparently.
Is the trial now binary or not? The market is always primed for strong efficacy results. But the power here is low and provide a fairly unreliable estimate for probability of success in P3. So ... no ... not really that binary. In other words what is more being tested is a futility type analysis with a low threshold for the no/go decision into P3.
Despite dozens of HC posts to the contrary discussing the first P2a results the efficacy informational value in that announcement was extremely low. Insufficient information was provided to allow a proper assessment – we all interpreted them simplistically (and wrongly).
This didn't matter, reflecting little actual real news the share price did very little. It was the rejection of the FDA BT (which was real news) that caused the nose dive. Both the unusual permeation testing method and the lack of strength in the efficacy outcomes led to the rejection. Legend in your own lunch time type stuff runs slap bang into real world.
None of us like living in the real world. We prefer more pleasant constructions to reduce anxieties. And so companies produce the makings for more positive ways of seeing things for investors. HC posters are the coalition of willing. We these positive reframes and with little intellectual hops and skips construct much happier places. These are very appreciated and we get many TUs for our troubles.
(20min delay)