fragile reversal, page-83

These are the main emails between Larry and I. I have left out a couple of small ones that didn't have anything of relevance in them. In some spots, i have included a comment. Comments are in capitals so you can see who is talking and where. The emails run in order from first to last.

Dear Mr Glass,

I contributed a series of posts to the Hotcopper forum questioning aspects of the fragile X data released to the asx by Neuren last week. I didn't intend to cause a fuss or to contact you about it, but I have been asked by several posters to forward the questions I have to you. I understand some others may have done this by copy/pasting my posts, but I thought I would outline my main concerns to you directly and ask for your opinion. In essence, my concerns relate to the statistical significance of the data.

Overall: what were the group sizes for the exps (number of mice) and how many repeats were done (ie separate experiments involving difference mice on different days). What was the statistical test used?

For graph 1 (open field rearing) the data looks good by eye. Why then were the differences between groups not tested for statistical significance? Was it because group sizes were too low?

For graph 2 (learning'), why is there no statistical testing shown between the treated and untreated frm ko groups? This is particularly important because this is the one comparison where you want to see a signif difference due to NNz-2566. Not that it particularly matters, but why show signif diff testing between treated WT and treated frm ko mice? It seems of limited importance, (particularly when compared to comparing treated v untreated ko groups which isn't shown), but even assuming it is important, it is then entirely reliant on the frm ko mice being different to the wildtypes, yet your testing of those groups shows no statistical significance. It would see to me that the results presented there are not supportive of an effect from the treatment.

For graph 3 (social behaviours), once again the presumably crucial comparison between treated and untreated frm ko mice is not directyl tested. Why is that? Fortunately, the untreated wt are found to be different from the frm ko, so that at least supports the use of this parameter. I also see that graph is described as trial 1. Was there only one exps performed with those mice? What does it mean that graph 2 does not show this (ie name them as Trial 1).

For fig 4 (neuron dendrites), one photo is shown for each group. Were more tested and was any objective quantification and statisitical analysis done (eg using software to quantify fluoresence).

I understand these are preliminary results, but they were described as significant which surprised me given the data shown.

FYI, I am a small shareholder and work and perform consultancy work in the biomedical area.

Kind regards,


REPLY

Please see our responses to your queries below. For convenience, we’ve copied your questions and provided our answers under each. Always happy to respond to questions from a shareholder (subject to ASIC disclosure guidelines, of course).

Best,


Larry


Overall: what were the group sizes for the exps (number of mice) and how many repeats were done (ie separate experiments involving difference mice on different days). What was the statistical test used?

Group sizes were N = 10 for all experiments.
(THIS SOUNDS GOOD< ALTHOUGH WOULD BE BETTER IF DONE ACROSS MORE THAN ONE EXP. MINOR QUIBBLE> BASICALLY FINE)


All animals underwent all behavioral tests. Tasks were performed in the order described with no more than one task performed per day. That is, there were no experiments split into multiple replications. All experiments were blind to the researcher performing the tests and the person injecting the mice.
(FINE)


Parametric data were analysed using two-way ANOVAs (genotype and sex as between-subject factors). Where data violated assumptions of normality or equality of variance, transformations (log10 or square root) were utilised. For repeated measures ANOVAs, homogeneity of variance was tested using Mauchly's test of sphericity and, where this was violated, Huyn–Feldt corrections were used. Non-parametric data were analysed using Mann–Whitney U tests. A p-value < 0.05 was considered statistically significant throughout.
(FINE, ALTHOUGH I HAVE ASKED ABOUT WHICH DATA NEEDED TRANSPORMING AND WHICH WAS PARAMETRIC OR NON PARAMETRIC. SOUNDS FINE THOUGH)


For ease of presentation on slides, the comparisons shown were limited to: effect of NZZ-2566 in Wild type mice (to detect a baseline effect of drug treatment), vehicle treated wild type compared to vehicle treated fmr1 knockout mice (to show abnormalities in the knockouts) and drug treated wild type with drug treated fmr1 knockout (to show drug treatment had normalized any abnormality). It should be noted that, although not shown, vehicle treated fmr1 knockout mice were also compared to drug treated fmr1 knockout mice to show a treatment effect. Also note that in all cases drug treatment statistically significantly reduced the abnormalities within the knockouts.


(MY QUESTION)For graph 1 (open field rearing) the data looks good by eye. Why then were the differences between groups not tested for statistical significance? Was it because group sizes were too low?

Differences were tested statistically – there just wasn’t enough room to plot every comparison at every time point on the slide. The difference between treated and untreated KO mice was statistically significant (p < 0.01). There were no significant differences among any of WT treated, WT vehicle and KO treated animals.

(MY QUESTION)For graph 2 (learning'), why is there no statistical testing shown between the treated and untreated frm ko groups? This is particularly important because this is the one comparison where you want to see a signif difference due to NNz-2566. Not that it particularly matters, but why show signif diff testing between treated WT and treated frm ko mice? It seems of limited importance, (particularly when compared to comparing treated v untreated ko groups which isn't shown), but even assuming it is important, it is then entirely reliant on the frm ko mice being different to the wildtypes, yet your testing of those groups shows no statistical significance. It would seem to me that the results presented there are not supportive of an effect from the treatment.

ANSWER
Comparisons were made between all the groups and selected results were shown for the sake of clarity. In all cases, treatment statistically significantly reduced the abnormalities shown in the NNZ-2566 treated gene-knockout mice compared to vehicle treated KO mice. In all but one measure, the effect of treatment was to make the gene-knockout mice statistically indistinguishable from normal mice. Complete normalization of behavioural and anatomical abnormalities seems a profound finding and therefore what we chose to report in the first instance. On the figure showing the learning experiment, there is a transposition error – the comparison performed show:



1. There is a statistically significant difference between the vehicle treated wild type and the vehicle treated knockout mouse (p < 0.01). This confirms the abnormal learning in the knockouts and should be shown on the Figure (THIS IS NOT WHAT IS ON THE GRAPH. THE OPPOSITE IS SHOWN - -THAT THERE IS NO DIFFERENCE. THIS MGT BE THE ERROR LARRY REFERRED TOO BUT I'M NOT SURE.)

2. There is no statistically significant effect comparing the vehicle and drug treated wild type mice, showing no baseline effect of drug

3. There is a statistically significant difference between the vehicle and drug treated fmr1 knockout mice, showing a treatment induced reduction of the learning deficit in the knockouts (THIS IS NOT SHOWN ON THE GRAPH)

4. There is no statistically significant difference between the drug treated wild type and fmr1 knockout mice, showing that the treatment effect present in 3) above is sufficient to normalise learning in the fmr1 knockout mice


(MY QUESTION) For graph 3 (social behaviours), once again the presumably crucial comparison between treated and untreated frm ko mice is not directyl tested. Why is that? Fortunately, the untreated wt are found to be different from the frm ko, so that at least supports the use of this parameter. I also see that graph is described as trial 1. Was there only one exps performed with those mice? What does it mean that graph 2 does not show this (ie name them as Trial 1).

ANSWER
Please see the discussion above relating to the choice of comparisons to present. For clarity:



1. There is no effect of drug treatment in the wild type animals compared to vehicle

2. There is a statistically significant difference between the vehicle treated wild type and fmr1 knockout mice, showing a social abnormality in the knockouts

3. There is a statistically significant difference between the vehicle and drug treated fmr1 knockout mice, confirming a treatment induced reduction of the abnormality in the knockouts

4. There is a significant difference between the drug treated wild type and fmr1 knockout mice. Despite the impression given by visual inspection, this is the one occasion when the significant reversal of the abnormality in the fmr1 knockout does not result in total normalization of behavior



“Trial 1” does not refer to, for example, first and second replications – all animals were tested on the same day per the answer to question 1. A second Trial 2 was conducted to attempt to detect an abnormality of social recognition. We are still analyzing these data, but it is not clear that social recognition was impacted in the knockout mice, or that there was an effect of treatment


(MY QUESTION)For fig 4 (neuron dendrites), one photo is shown for each group. Were more tested and was any objective quantification and statisitical analysis done (eg using software to quantify fluoresence).

ANSWER
Hippocampal cell cultures were prepared from wild-type and fmr1 KO foetal mice (14 –16 d of gestation). Briefly, mice were sacrificed by cervical dislocation under chloroform anaesthesia, and dissociated hippocampal cells were plated in 15 mm multiwell vessels (Falcon Primaria). A plating medium of MEM-Eagle’s salts (supplied glutamine free) supplemented with 10% foetal bovine serum, was used. Cultures were kept at 37°C in a humidified 5% CO2 atmosphere.


After 3d in vitro, green fluorescent protein (GFP) was used to monitor dendritic spine morphogenesis during time-course of culture using standard computer automated image analysis techniques (Ethell and Yamaguchi, 1999; Ethell et al., 2001, Henkemeyer et al., 2003). The dendritic spines are usually formed between 7 and 14 days in vitro (DIV). By 14 DIV most dendritic protrusions are spines; however, their maturation continues until 21 DIV. We used a compartmentalized culture system, a microfluidic chamber which opens the possibility for fast drug testing with the capacity to detect in vitro drug effects such as spine morphology, neurite outgrowth and synapse formation. Please see question 1 above for statistical techniques.
(THIS IS OUTSIDE OF MY AREA OF EXPERTISE, BUT FROM WHAT I KNOW ABOUT THIS TYPE OF ANALYSIS, IT SOUNDS LIKE IT WAS DONE PROPERLY. THE MAIN THING I WAS INTERESTED IN WAS THAT IT WAS DONE OBJECTIVELY (IE BY MACHINE AND WITHOUT POTENTIAL USER BIAS). IT SOUNDS LIKE ALL WAS DONE PROPERLY.



Quantitation showed NNZ-2566 reduced the number of excess dendritic spines in a dose-dependent manner, to a level comparable with vehicle treated neurons from wild type animals. These data are being prepared for publication.

Larry Glass


MY REPLY

Dear Larry,

thank you for the quick and detailed reply. i have a couple of questions:

do you mind if I publish the answer o the hotcopper forum? As discussed, I have only contacted you because I was asked to do so, but I want to be sure you are ok with me publishing your answers.

Second, the written answers you provide are reassuring, but do not match the figures in the asx release. You mention all groups were compared and acknowledge a tranpositition error in graph 2, but in fact, there is no comparison shown between treated and untreated KO mice at all in graph 2 or graph 3. Only 3 comparisons are shown for each of those graphs, none for treated versus untreated ko mice. Rather than a transposition, have they simply been left out? If so (as appears likely from your answers), will a correction be made (I'm not requesting this - jsut anticipating questinos from the forum). if there is a transposition, are the comparisons that are shown on the graphs in the release correct?

Lastly, re the stats, it seems you've applied parametric, transformed parametric and non parametric testing. I assume that for each set of data only one approach was applied. Could you let me know which is which. ie which data set was parametric, which non parametric, and which needed to be transformed? Actually, kind of surprised you did 10 mice in one hit. Would ahve been nice to see it over more than one exp (even wth same numbers ie 2 x 5). That's not a question, just an observation about something i found odd.

Thank you again for your help. It is much appreciated.

Kind regards,


REPLY

Yes, please feel free to share our response. I would appreciate your sharing the complete response. Our responses to your question refer to the analyses that we conducted rather than to what we showed on the slides in the presentation. As we noted, we chose to emphasize normalization of fmr1 KO mice rather than differences between NNZ-2566 and vehicle treated KO mice but, as indicated in the response, the latter comparisons were also statistically significant. (IT DOES SURPRISE ME THAT THESE WEREN'T SHOWN) We don’t plan to release a complete report outside of a scientific meeting or journal so, basically, you’ll have to take our word for it (THAT IS FAIR ENOUGH, ALTHOUGH THEN THERE WILL BE QUESTIONS THAT CAN;T BE RESOLVED UNTIL THEN TOO OF COURSE. NO BIG DEAL THOUGH.). If you look at the NNZ-2566 versus vehicle treated data for the KO mice in the three slides, I don’t think you’ll have trouble imagining that the effect was significant. (SHOULD NOT REALLY NEED TO IMAGINE, BUT I THINK HE'S JUST SAYING THAT THE STATS LINE UP WITH THE APPEARANCE OF THE DATA). I’ll have to query the statistician on which analytical method was used for which experiment. With respect to running 10 mice per group at a time, these are very labor intensive experiments so it’s much more efficient to set them up and run them all at once. Remember that this was our first FXS study and what we were looking for was a signal, not Phase III clinical trial grade statistical significance. What we got was more profoundly positive than we dared hope and, as Joe said in the announcement, it’s very exciting.

(I HAVE A FEW QUESTINOS THAT I AM CHASING UP, MAINLY ON FIG 2. WILL POST ON HC WHEN I HEAR ANYTHING BACK)
Again, thanks for your interest. Shareholder exuberance (which, of course, we share) notwithstanding, honest skepticism is healthy and productive as well.


Best,


Larry