Is Gøtzsche good or bad for science? Bastians, Greenhalghs and Moynihans vs Healys, Smiths and Godlees. The radicality demanded by the «GIGO effect». By Abel Novoa

Is Gøtzsche good or bad for science?

That’s what Hilda Bastian asks herself in a recent post on her wonderful blog «Absolutely maybe».

Hilda Bastian was also founder of Cochrane, like Gøtzsche, but left the organisation in 2012 because of some discrepancies with its strategic approach, similar to those referred to by Gøtzsche in his letter after his expulsion:

«I agree with him on several counts – I finally parted ways with the Cochrane Collaboration in 2012, because of the commercialization, open access issues, and the strategic direction it chose. There are real problems, although there are obviously enough people who want it this way.»

Boilover means boiling overflow and is a very dangerous situation, feared by firefighters, that occurs when containers of combustible liquids are set on fire. This is what she believes is happening and calls the prudence of the firefighters.

Probably she fears to take the causes of its march of Cochrane to its last consequences: it is not only that it is problematic that the reviewers of Cochrane have conflicts of interest with the manufacturers of the products that they evaluate is that this minimum safeguard – that is to say, although all the reviewers Cochrane were a Gøtzsche – would follow without being able to neutralize the effects that the conflicts of interest are having on all the system of generation, synthesis, diffusion and application of the biomedical knowledge.

A Cochrane without conflicts of interest, without a commercial focus, without a dependence on donors with clear ideological agendas, would continue without being able to counteract the GIGO effect: Garbage In – Garbage Out. That`s the problem

Hilda advocates a more independent Cochrane but accepts that the knowledge provided is not. That’s why her «technical» criticism of Gøtzsche and his gang, although it may be true, is irrelevant. His «political» criticism, on the other hand, is relevant, although demagogic and not too honest, I think.

 

It is clear that Peter Gøtzsche is not one of the people he leaves indifferent neither in his personal treatment nor in his way of producing, criticizing and disseminating science but, in our opinion, his attitude is coherent with the GIGO effect to which he has dedicated almost all his life as a researcher and writer.

The GIGO effect can only have radical consequences even if one, like me, is not a radical.

Let’s see what Hilda Bastian says.

Hilda has been belligerent in some aspects of Jørgensen, Gøtzsche and Jefferson’s (hereinafter JGJ) critique of the Cochrane review of the HPV vaccine. For example, he finds the allegation that half of the existing studies were not included in the review or that there was a serious underestimation of adverse events lacks rigour:

«Publishing a claim that half the eligible trials and participants are missing required references…When I started fact-checking the critique, it took only minutes of fact-checking to see that some of the criticisms were errors. It took only the normal effort level of peer review to see the serious error of saying Cochrane and the trialists had under-reported the number of serious adverse events in a trial, when what they reported was in fact the number of women experiencing adverse events – which is of course smaller.»

For this reason, he accuses the editors of BMJ EBM of not having carried out a sufficiently careful work of revision of the text of JGJ (we dedicate to this matter an entry in NoGracias) and lets fall that the friendship of one of the editors in chief of BMJ EBM, Carl Heneghan, with Tom Jefferson, one of the critical authors, has been able to influence since, in addition to friend, «it is a defender of the same approach of the methods of systematic revision. (our bold)

Same approach? What do you mean by that expression?

Heneghan is an author quite aware of the GIGO effect and consistent with what it implies to assume it. For example, he is co-author of this BMJ editorial that denounced the existing uncertainty regarding the safety of new anticoagulants and that neither the large clinical trials carried out nor the elegant systematic reviews and meta-analyses that synthesised them were capable of detecting:

«We have to find ways to reduce uncertainty and improve knowledge about the risk-benefit balance of these drugs.»

This BMJ editorial went in the same direction as a BIT Navarre monograph that denounced the fragility of the evidence behind some medicines, the new anticoagulants, with great potential for expansion and high risk.

Juan Erviti, Luis Carlos Saiz and Javier Garjón, linked to the BIT, are also very aware of the GIGO effect and act accordingly: they say, we don’t trust and we don´t want clinical trials or meta-analyses or evaluations of regulatory agencies that manage knowledge with probable biases and corroded by conflicts of interest, economic and political, of their agents, and we put the safety of the patients ahead of the commercial interests of companies and scientific societies» (in NoGracias we comment on the attempt to intimidate the scientific societies bought by the industry that went into a rush to defend «serious science»). 

Does Hilda mean this coherence that has to accept the GIGO effect with the expression «same approach»? It seems not. Same approach means for Hilda «same ideological prejudices» and, for her, Gøtzsche and her (fortunately ever wider) gang are trying to hide their ideological prejudices under their demands for freedom of scientific opinion:

«There are limits to all freedoms – right where they butt up into responsibilities. You’re only communicating a “fact” if it’s actually true. That requires serious diligence. Some ideas are extraordinarily damaging, to individuals and society, and have no redeeming value. … there are prerequisites for healthy debates.»

Obviously neither Heneghan nor Jefferson nor Gøtzsche are infallible and can be wrong. It is also possible that, without equivocation, the interpretation of which studies are eligible for inclusion in the meta-analysis is debatable. Trisha Greenhalgh put it well in her text about this crisis that we translated and commented on:

«The philosophical explanation is that facts are not self-interpreting; they are theory- and value-laden. Even when there are agreed criteria for including or excluding a study or for assigning a particular score to an aspect of the methods, multiple subjective judgments need to be made…Because of the need for judgement on such questions, two systematic review teams can produce different findings even when both teams are expert and use identical checklists and statistical methods.» 

But any technical discussion of a systematic review or meta-analysis that is not contemplating the GIGO effect is «gardening the pharmaceutical industry. Accepting the GIGO effect is redeeming. To discuss technical aspects of «the garbage that comes in» is to participate, collaborate and contribute to the continuation of the game that produces the GIGO effect.

The problem, in my opinion, is that although Hilda Bastien was right about which clinical trials were eligible, the clinical trials chosen have a high probability of being biased and therefore biased will be any Cochrane review that includes these trials.

It is a question of courage but also of rigour. Hilda Bastien (also Trisha Greenhalgh or Ray Moyniham) know that industry-sponsored clinical trials, directly or indirectly, are rubbish. Today there is no system of knowledge governance capable of preventing the biases that commercial research introduces in all the processes -ethical, methodological, statistical or logistical- involved in conducting a clinical trial.

Everything is infected, as Jacob Stegenga describes well in his Medical Nihilism: the criteria for defining disease, the development of instruments for measuring results, the control of Research Evaluation Committees (especially in developing countries where most experiments are carried out today), the recruitment of collaborating researchers, the design of the experiment, the recording of data, statistical analysis, the writing of scientific articles, the control of publishers (and publishers) who publish the texts, the evaluation processes of regulatory agencies, scientific societies and experts who must position the product in the Guides and in clinical practice…everything is tied up and well tied up.

I think it is a scandal that Bastian calls Heneghan, Jefferson or Gøtzsche «merchants of doubt» and compares them with the scientists who serve corporations in order to protect their economic interests through junk science: the latter for money; the former for ideology:

«Ideological and commercial “merchants of doubt” are causing serious damage, and we’re not good at dealing with it yet.»

Not only that. Bastian compares the behaviour of Heneghan, Jefferson or Gøtzsche (surely anyone who assumes the GIGO effect) with that of a swindler like Wakefield – who linked the triple virus with autism – and claims, in a demagogically apocalyptic way, that these authors will be guilty of the public health problems that their criticisms may bring, as Wakefield is guilty of the emergence of measles in Europe.

If the comparison between a swindler and accredited fallible scientists is already odious, the fact that there are lives at stake and that open criticism of science goes against public health is an improper demagogy of an author we respect as Hilda Bastian.

If critical authors like Bastian or Greenhalgh fall into the ideological trap, what will become of the gullible?

Against the crisis of the Cochrane there are only two exits. Either the GIGO effect is accepted and we continue to feed a broken system (which is what Bastian, Greenhalgh or Moynihan seem to want) or the GIGO effect is not accepted and we act accordingly: not a single step backwards and several forwards.

No doubt there is a political disarmament of those who assume the GIGO effect: they can easily be accused of being against industry, sold scientific societies, scientists and opinion leaders with conflicts of interest, for being anti-capitalists; they can also be accused of being against the current policies and strategy of the Cochrane Collaboration for being anarchists or vain egomaniacs; and, of course, they will be accused of anti-scientists and even potential homicides as Hilda does.

What about those who assume the radical consequences of the GIGO effect but do not look like anti-capitalist radicals, egomaniacs, magicians or murderers? Are they all swindlers who put their biases at the service of their ideologies and do not care about the consequences for public health? What about the GIGO effect that Bastian, Greenhalgh or Moynihan have contributed to unveiling with their excellent work? Why are they not taking its effect to its ultimate consequences?

It is clear that to assume the GIGO effect and its consequences requires courage, commitment and, in the current scenario, to accept, surely, a certain degree of ostracism. In this scenario, to ask for responsibility is to demand radicality, not damage control, tranquillity or institutional governance.    

Richard Smith, ex-editor of the BMJ does assume the GIGO effect and summarizes it in the title of his entry on the subject: Is Peter Gøtzsche the boy who sees that the emperor has no clothes and says so?

In this text, Smith reproduces the introduction he made to Gøtzsche’s book, «Medicines that kill and organized crime,» It begins like this:

«There must be plenty of people who shudder when they hear that Peter  Gøtzsche will be speaking at a meeting or see his name  on the contents list of a journal. He is like the young boy who not only could see that the emperor had no clothes but also said so. Most of us either cannot see that the emperor is naked or will not announce it when we see his nakedness, which is why we badly need people like Peter. He is not a compromiser or a dissembler, and he has a taste for strong, blunt language and colourful metaphors.  Some, perhaps many, people might be put off reading this book by Peter’s insistence on comparing the pharmaceutical industry to the mob, but those who turn away from the book will miss an important opportunity to understand something important about the world—and to be shocked.»

David Healy not only accepts the radical consequences of the GIGO effect and has launched an entire independent system to determine the side effects of drugs, but accuses the Cochrane (or NICE) of negligence for not doing so:

«Finding that trials yield inconsistent results should result in a statement that we don’t know what we are doing  – clinicians and the public should beware any claims to the contrary.  Inconsistent trials should not result in NICE guidelines mandating treatment approaches from HPV vaccines to antidepressants – especially when the inconsistencies are based on surrogate outcomes, from scores on a Depression Rating Scale to bone densities, rather than outcomes that count to patients. If it weren’t for Cochrane nobody would have to make blindingly obvious statements like this.»

Cochrane takes years taking care of the garden of the pharmaceuticals although at the beginning it seemed a critical instrument: 

«Many engaged with Cochrane, thinking it was a way to contain the pharmaceutical industry. When it began, Cochrane had a chance to bring industry to heel by saying it would only include trials and treatments where the data was available in its reviews. The Collaboration was in a better place to force industry to engage with science than anyone else was.  It blew this opportunity then and has continued to sell its birthright ever since… Cochrane at present is one of the biggest obstacles there is to people getting Data Based Medicine.»

Healy’s still with his hammer: 

«The evidence that pretty well all the clinical trial literature on on-patent pharmaceuticals is ghostwritten has been around since 2000.  Senior figures in Cochrane and others who go about meta-analysing trials know this but have ignored it.»

Cochrane ignores the effect GIGO and prefers elegant methodological dissertations on the garbage that revises:

«Cochrane make a great deal of noise about applying a range of indicators to publications – from clarity of randomization procedures to conflict of interest statements – in an effort to appear rigorous or methodical.  Peter Goetzsche was one of those who contributed most to the development of a range of instruments of this sort.»

This complicated methodology has the opposite effect to that sought: it makes it very difficult for non-professionals, for those who do not know in detail the doctrine of evidence or do not have time or do not believe in it, to be able to generate knowledge that is considered relevant by the church of evidences:

«These methods may fool some into thinking industry trials have been appropriately sterilised and can be used but rather than advancing science, they have a primary effect of excluding the laity. n multiplying the tick-boxes around trials, these processes have created a strange Alice in Wonderland type of needle’s eye through which industry camels can swagger to publication in NEJM, JAMA or the Lancet or get into a Cochrane HPV vaccine review, but anyone attempting to report a significant adverse effect on a drug or a vaccine finds it impossible to get published.»

The ghost writers hired by the industry are of course well acquainted with the doctrine of evidence and turn scientific articles into incontrovertible and unattainable texts in their standards:

«The one thing about ghosts is that they ensure  publications come with all boxes and quality indicators ticked in a way that the average clinician or researcher cannot hope to emulate».

There is no worse published science than biomedicine. There is not so much fake news in any other area of knowledge as in biomedicine and Cochrane has played a key role in ensuring that this «fake system» continues.

Recently, Tom Jefferson along with Lars Jorgensen also denounced the GIGO effect in a BMJ editorial::

«Our reliance on journal articles needs a redefinition, if not a shift. In the last decade, evidence has accumulated, across a spectrum of different interventions, that journal publications cannot be trusted. Article reports of clinical trials suffer from a grave illness which is curable, but needs a concerted approach to prevent the growing threat of reporting bias.»

The text suggests that Cochrane reviewers should work on the basis of Clinical Study Reports (CSRs) that companies submit to regulatory agencies and ignore articles written by professional writers paid by industry, even though they have all the quality criteria.

As Healy denounces:

«The recent HPV review appears to have worked from ghostwritten publications rather than CSRs.»

But even these CSRs are not reliable, as Jefferson and Jorgensen point out. Direct access to databases is needed.

Healy speaks the hypocrisy of ignoring the GIGO effect:

«Journals from the BMJ to the NEJM know this, as do academics – many on some of the listserves to which this post will go – but they nevertheless extol Cochrane and mislead others into thinking Cochrane is independent – as it would be if it had access to the data from trials.»

Healy has clear that the Cochrane collaboration does not respond to the objectives of Archie Cochrane:

«Cochrane Inc. began just after Archie Cochrane died in 1989.  For Archie C, trials were a means to combat medical arrogance and therapeutic bandwagons – such as coronary care units when these began in the 1960s.»

Archie Cochrane thought that the trials would discredit the medical, technological and pharmacological interventions that were being incorporated into medical practice without control. The trials should not be neutral but instruments to de-medicalise, improve the efficiency of health systems and control the interests of professionals and companies, but: 

«Immediately after his death, RCTs in the hands of the Cochrane Collaboration, were reborn as value neutral.  There was no longer any need for doctors or others in healthcare to bother their pretty little heads (the men that is) about what healthcare should look like.  Their job was follow the trials and apply what worked and healthcare would take care of itself.»

All of this doctrine and approach was sold, among others by the Cochrane Collaboration, as Evidence-Based Medicine rather than Eminence-Based Medicine:

«The message was trust us not them.» 

If the Cochrane Collaboration had opted to commit itself with the data, this allegation could have certain value. But it preferred to accept the effect GIGO with the known consequences:

«What the HPV vaccine story reveals is that Cochrane can’t be trusted.  RCTs and Systematic Reviews have become the fuel of therapeutic bandwagons rather than a means to derail these bandwagons.»

BMJ director, Fiona Godlee, also believes that the problem is that the current management of Cochrane has accepted to be too close to the industry and is a situation that can not go on like this:

«The situation is still evolving, and details are still emerging, but the governing board’s vote to expel one of its founders and most vocal internal critics, Peter Gøtzsche, brings to a head years of growing tension between the collaboration’s radical academic roots and its more recent corporate identity. The board’s statement cites bad behaviour, but beyond the personalities lies a deep seated difference of opinion about how close to industry is too close…But he calls for a ban on financial conflicts of interest among Cochrane reviewers. The BMJ supports this call. It would mean fewer but better systematic reviews.»

It is true that there is a problem with Goetzsche («personalities») when he raises much of the problem as a corruption issue and accuses people with conflicts of interest directly of being immoral. This is a weakness of Peter’s discourse. The problem has nothing to do with individual behaviour: there is corruption, no doubt, but not enough to explain what is happening.

We have called it «institutional drift»:

«A situation that occurs when secondary interests modify the objectives of medicine, through a systematic influence that alters routines and transforms the culture of the organization and the behavior of the agents, with consequences that are difficult to identify, due to unconscious, socially accepted and/or legal behaviors.»

And it is the consequence of the mode 2 innovation system (Gibbons) or of the post-academic science paradigm (Ziman).

Javier Echevarría directly calls it a techno-scientific revolution and it happened in the 80s, establishing a new context in the production, synthesis, diffusion and application of knowledge.

The Cochrane crisis is no more than another twist of an economic triumphant system and profoundly harmful in scientific, professional, political and cultural terms.

Goetzsche would do well to avoid personal attacks and his supposed moral superiority and assume a position more «Political» than war against the corrupt. 

And I finish.

Godlee calls, like Healy, to recover the foundational ends:

«We must hope that Cochrane remembers its roots, and that it comes through this episode reinvigorated, independent, and committed to holding industry and academia to account.»

The question would be:

All -Jefferson, Jorgensen, Goetzsche, Smith, Healy, Heneghan, Erviti, Gérvas, Laporte, Godlee or the four Board members who have resigned (Joerg Meerpohl, Director of the German Cochrane Centre; Gerald Gartlehner, Director of the Austrian Cochrane Centre; Nancy Santesso, Super User, Canadian Cochrane Centre, David Hammerstein, civil society advocate for the Commons Network) and many more-are all «merchants of doubt» and dangerous enemies of public health, Hilda?

Improving the governance of the Cochrane (eliminating reviewers with conflicts of interest, avoiding donors with ideological agendas, accepting internal debate, preventing the mercantilist drift of the organization …) is only the beginning.

This must continue with the challenge by the entire scientific, professional and citizen community of a system of production, synthesis, dissemination and application of knowledge undermined in its foundations and structure by the market, economic interests, management strategies of health and scientific organizations and the ideological agendas of governments.

As we said elsewhere:

«Guaranteeing professional independence is not primarily an individual medical problem, but an institutional and social one. Strong professional institutions are needed, committed to the protection of medical judgement and objectivity of knowledge; they also need to activate political instruments. Only through a system of mutual reinforcement (professional self-regulation plus legislation) will it be possible to break the dynamics of conformity that are causing the drift of medicine.» 

It is not ideology but democratic politics and professionalism:

«Limiting the market and the growing power of organizational bureaucracies is not an ideological matter, but clearly a professional one. Interested accusations of ideologization against professional independence initiatives are putting patients and populations at risk in the name of particular interests.» 

I have no doubt about the intentions of the magnificent authors I criticize in this post. My position is contrary to their tempered solutions (and in the case of Hilda Bastien, their inferences that I believe are demagogic) that will only perpetuate the status quo.

 

As Marc Casañas, of the Board of Directors of NoGracias says, and that José Valdecasas, of the Advisory Commission of Nogracias, supported in twiter:  

«Let gasoline rule»

(#TargaryenOnMode)

The GIGO effect makes many radicals

Abel Novoa is president of NoGracias