Skeptical of medical science reports?Copyright and/orpublishing...

Skeptical of medical science reports?

Copyright and/orpublishing rights held by the Canadian Veterinary Medical Association

“It is simply no longer possible to believe much of theclinical research that is published, or to rely on the judgment of trustedphysicians or authoritative medical guidelines. I take no pleasure in thisconclusion, which I reached slowly and reluctantly over my two decades aseditor of The New England Journal of Medicine” (1).

More recently, Richard Horton,editor of The Lancet, wrote that “The case against science isstraightforward: much of the scientific literature, perhaps half, may simply beuntrue. Afflicted by studies with small sample sizes, tiny effects, invalidexploratory analyses, and flagrant conflicts of interest, together with anobsession for pursuing fashionable trends of dubious importance, science hastaken a turn towards darkness” (2).

The first of these two commentarieson clinical research publications appeared in 2009, the second in April of thisyear. These statements are being taken seriously, coming as they do from theexperiences of editors of two of the world’s most prestigious medical journals.The first article showed how the relationships between pharmaceutical companiesand academic physicians at prestigious universities impacted certaindrug-related publications and the marketing of prescription drugs. Potentialconflicts of interest seemed to abound: millions of dollars in consulting andspeaking fees to physicians who promoted specific drugs, public researchdollars being used by a researcher to test a drug owned by a company in whichthe researcher held millions of dollars in shares, failure of university researchersto disclose income from drug companies, company subsidies to physiciancontinuing education, publishing practice guidelines involving drugs in whichthe authors have a financial interest, using influential physicians to promotedrugs for unapproved uses, bias in favor of a product coming from failure topublish negative results and repeated publication of positive results indifferent forms. The author, Marcia Angell, cited the case of a drug giant thathad to agree to settle charges that it deliberately withheld evidence that itstop-selling anti-depressant was ineffective and could be harmful to certain agegroups (1).

Marcia Angell’s comments (1)were directed largely against conflicts of interest and the biases introducedby the influence of drug companies on researchers and universities. RichardHorton’s statement (2)was part of his comments on a recent symposium on reliability andreproducibility of research in the biomedical sciences and addresses a broaderarea of concern. Some of the problems he identified are seen in the veterinaryliterature. They include inadequate number of subjects in the study, poor studydesign, and potential conflicts of interest. He notes that the quest forjournal impact factor is fuelling competition for publication in a few highreputation journals. He warns that “our love of ‘significance’ pollutes theliterature with many a statistical fairy-tale” and he remarks that journaleditors, reviewers, and granting bodies all stress original studies to theextent that “we reject important confirmations” (2).

Individuals and organizationsconsidered responsible for the present state of published medical scienceinclude researchers, journal editors, reviewers, granting agencies,governments. Horton goes on to reflect on whether the bad practices can befixed (2).He concludes that scientists have incentives to be productive and innovativebut no incentives to be right. He muses on removal of incentives, emphasizingcollaboration rather than competition, improving research training andmentorship, funding studies that attempt to replicate published data. Hortonended by noting that it is a good first step to recognize the problems but noone seems ready to begin the task of reversing the trends.

Clinical journals such as TheCanadian Veterinary Journal are less affected by the fight for the impact factorbecause the primary impact that we seek to make is on the clinical practicecommunity, rather than the research community (the journal impact factor isbased on the impact on the researcher community). Nonetheless, we share some ofthe problems discussed above. Perhaps the most serious weakness is inadequatesample number in some studies. Such studies are sometimes accepted because theymay have some value if care is taken to acknowledge the limitations associatedwith inadequate power. The take home message is that readers must exercisecaution in interpreting the published literature, regardless of the reputationof the journal in which an article is found.

References

1. Angell M. Drug Companies &Doctors: A Story of Corruption. The New York Review of Books magazine. [Lastaccessed August 5, 2015]. Available from: http://www.nybooks.com/articles/archives/2009/jan/15/drug-companies-doctorsa-story-of-corruption/

2. Horton R. Offline: What ismedicine’s 5 sigma?