Leprosy Mailing List – January 5, 2020
Ref.: (LML) Rapid, open scientific discourse for improved outcomes
From: Joel Almeida, London and Mumbai
Dear Pieter and colleagues,
Richard Smith, formerly editor of the British Medical Journal, has reasonably well-informed insights about the scientific method and the role of traditional publication. Printed journals stored in physical libraries were once the only option. However, Smith in 2006 described what he considered to be a preferable approach:
"Open up the whole process and conduct it in real time on the web in front of the eyes of anybody interested. Peer review would then be transformed from a black box into an open scientific discourse." (1)
What prompted Smith to take this view?
In his words: "the phrase `peer reviewed journal' is supposed to guarantee quality. But clearly peer review is deficient. Despite being central to the scientific process it was itself largely unstudied until various pioneers—including Stephen Lock, former editor of the BMJ, and Drummond Rennie, deputy editor of JAMA—urged that it could and should be studied. Studies so far have shown that it is slow, expensive, ineffective, something of a lottery, prone to bias and abuse, and hopeless at spotting errors and fraud. The benefits of peer review have been much harder to establish. As Rennie says, `If it was a drug it would never get onto the market'." (1)
Traditional journals, despite all their merits, too often have sheltered errors unintentionally. That's because it's easier for 2 reviewers inadvertently to overlook errors than for hundreds of experts to fall short in this way. The results section of most papers is often reasonably reliable. However, even when the methods are sound, the introduction and the discussion sections allow considerable latitude for error. Thereafter, the paper can gather dust on a shelf and its flaws can remain unnoticed and uncorrected. That is why it is important to look behind the abstract or the introduction /discussion /conclusions and examine the raw observations in any paper of consequence.
Scientific method vs extraneous considerations
Non-scientific criteria can sometimes creep into scientific discussions. For example, if we go back in history we come to Galileo and his claim that the earth revolved around the sun. How did the traditionalists respond to his work "Dialogue concerning the Two Chief World Systems" (3)? They asserted that the typeface of Galileo's work was inappropriate, and the organisation of his document was different from a scientific book. These objections were extraneous to the scientific method. They are similar to discussions about where something is written. Galileo had the truth. In the end, only the truth works.
The scientific method clearly is indispensable to science. The traditional journal, despite its unique role in the pre-internet era, is less so. We need to keep faith with the scientific method: conjecture and refutation.(4) But traditional journals are no longer the only available channel in the internet age. Further, traditional journals as well as other channels are reliable only to the extent that they weed out error.
Meetings too can play a useful role, although the travel and time involved tend to limit participation compared to online discussions. At a meeting of the German Paediatric Society at Kassel in October 1960, Kosenow and Pfeiffer presented evidence of two infants showing malformations. It would have been unwise and dangerous to neglect this first report of the apparent adverse effects of thalidomide in pregnant women merely because it was made outside a traditional journal. Typefaces, venues etc are not part of the scientific method. It seems wise to rely on the scientific method and not on extraneous considerations.
Solutions
LML, like Gates Open Research, Wellcome Open Research, UCL Child Health Open Research, enables rapid publication of contributions that are exposed to open scientific discourse of the sort described by Smith.(1) The author(s) can then reconsider, refine and correct their analysis. This interactive cycle progressively and relatively rapidly eliminates errors. On LML, contributions are read (and substantive disagreements, if any, identified) by a larger set of knowledgeable experts than just the 2 reviewers typical of traditional journals. That is why errors tend to have such a short half-life on LML. As a result, the truth emerges more rapidly in such online channels than almost anywhere else. The Gates Foundation and the Wellcome Trust are not naive in their choice of publishing strategy. They reinforce the direction of travel.
Traditional journals are also characterised by slowness. The sheer speed of the LML process is unrivalled. Decisions on contributions are made in days, not months (or sometimes even two and a half years in one case involving an HD journal, which ultimately still failed to reach a clear decision). Time is of the essence because human nerves, limbs, eyes, minds, livelihoods and relationships are at stake.
LML contributions are probably the most widely read in the HD world, partly because everyone has a voice here and can put forward substantive points immediately. The contributions can then be stored, displayed and searched online for anyone to view. They can be, and have been, cited in some of the most fastidious traditional journals. If and when fashions steer us towards a cliff, such open forums sometimes can help remind us of the difference between right and might. As a result, we are less likely to expose the people we serve to unnecessary harm.
Evolution of effective policy has always benefited from wide discussion. This was true even before the internet. In the UK, such wide discussion is now required by the National Institute of Health Care and Clinical Excellence (NICE) before guidance can be issued. The NICE process is more open and error-resistant than some others. Discussion in the NICE process is carried on outside the traditional journals. After NICE guidance is issued, it is kept under constant review as new information surfaces. In this way, harmful guidance is weeded out and effectiveness is constantly maintained or even boosted. In HD, LML is the nearest we have to the NICE process for improving effectiveness and outcomes.
A brighter future
Having been in the thick of the struggle to help transform TB from a neglected disease in the early '90s to a major global issue with boosted financing and life-saving outcomes, I believe that the kind of pragmatic action-oriented analysis often found on LML is likely to be reasonably impactful. As my mentor the great Dr. Kochi used to say at WHO, "Make complex things simple if you want great outcomes." At the Global TB Programme we relied on well-researched, persuasive reports that were often acted on by health ministries, finance ministries, planning commissions, foreign aid departments, the World Bank, NGOs, private physicians and the media. The real world is messy, and we must still discern the few critical actions that enable great outcomes.
Fortunately, we are a compassionate community with a clear memory of why we are involved with HD at all. We have tasted real success and now it is unlikely that we will settle for less. Since Shandong and Uele achieved 17% to 20%/year declines in HD, so can we. Since the Brazilian programme and roving paramedical workers in Dadra Nagar Haveli (India) monitor nerve function regularly, so can we. Since the world considers it important to protect vulnerable people from unintended adverse effects, so can we. LML helps all this greatly.
Joel Almeida
References
1. Smith, R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006 Apr; 99(4): 178–182.doi: 10.1258/jrsm.99.4.178
2. Peters D, Ceci S. Peer-review practices of psychological journals: the fate of submitted articles, submitted again. Behav Brain Sci 1982;5: 187-255
3. Galileo Galilei, Stephen Jay Gould (Editor), Stillman Drake (Translator), Albert Einstein (Foreword), J.L. Heilbron (Introduction). Dialogue Concerning the Two Chief World Systems. Modern Library, 2001. translated from: Dialogo sopra i due massimi sistemi del mondo. Florence, 1632.
4. Popper, K. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge; second edition, 2 May 2002
LML - S Deepak, B Naafs, S Noto and P Schreuder
LML blog link: http://leprosymailinglist.blogspot.it/
Contact: Dr Pieter Schreuder << editorlml@gmail.com
You received this message because you are subscribed to the Google Groups "Leprosy Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leprosymailinglist+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/leprosymailinglist/f326cfaf-ec33-498d-9526-2007de37c770%40googlegroups.com.
No comments:
Post a Comment