In short, the 60% drop out rate from the already small subject pool means that, imo, this study is not really reliable. If it is repeated and similar results are found, then that will add to its reliability. A larger cohort would have also added more reliability to the study results. The study not really being valid, again, largely stems from the 60% drop out rate combined with the results not quite matching the claims. Not meaning to insult anyone's intelligence, but flipping a coin 3 times and getting 2 heads does not mean that a coin flip has a 66% chance of landing on heads. Hence the need to have larger numbers in a RCT to weed out any randomness that may skew or bias results. Further to those points - "Our experience as a tertiary brain center has demonstrated that there often is difficulty for patients to return for testing after 3 months, especially if a treatment was considered to be successful". We are expected to take Carrick at his word here. No reference or citation given and it's a rather convenient excuse for the high drop out rate. How do we know in this case that patients dropped out because the treatment was successful? How do we know it wasn't a lack of desire to return for more uncomfortable and stressful tests if there was no real benefit from treatment. Subjects in trials like this may be too kind to be blunt and say the therapy didn't work, or, more likely, they weren't asked for their thoughts. A lot was made of the qEEG testing as a confirmation of ASD profile and measure of effectiveness, yet only 24 (28% of initial cohort) participants were included in the qEEG follow up. It's a long bow to claim any real statistically significant changes in such poor follow through numbers, as the authors attempt to do in the conclusion. I couldn't understand the focus on posturogrpahy throughout the study. References to how robust the CAPS measurement tool was in handling the "abuse" it took from some subjects and how accurately it could measure "very small changes" only made sense when I read the author disclosures and saw that 2 authors "have financial interests in Vestibular Technologies, LLC, the company that manufactures and sells the CAPS Professional System". This study wasn't meant to be about the NFB device, not the posture device, and it reads as if they are also trying to validate the CAPS system in ASD diagnosis. The questionnaires results also seemed to be all over the place and do not really match the conclusions made. The questionnaires themselves are valid, but I think the results suffer from such a low cohort number and seem to be biased by a couple of really strong results in the active arm that skew the box plots. The BRIEF questionnaire also removed 5 participants due to variability of their scores, making the results even less reliable. There was also a worsening of some key ASD diagnostic indicators, namely speech and language (ATEC questionnaire) despite parents self reporting and improvement here. This is a common bias for parents, thinking their children are doing or responding better than they actually are. The paragraph on qualitative reporting is pointless without any examination or data to support it. This report could be biased by the timing and type of qualitative questions asked. The final key point to all of this though, is there is minimal statistically significant evidence to support the device at this time, based on this study. That does not mean the device is clinically significant. Imo, it looks like it might work well in some ASD patients, but even that is hard to quantify from this study. More likely, the device is helpful in reducing ADHD comorbid factors, which is one area where there is some evidence of efficacy for NFB interventions. Hope that helps.
Expand
(20min delay)