Regenerate Cochrane to strengthen the production of trusted evidence for the common good of public health. By David Hammerstein

The crisis in Cochrane is about the credibility of Cochrane and not a  question concerning the “behaviour” by one individual. What is at stake is the prestige of Cochrane and the public´s faith in its work. It is a major mistake to personalize the crisis in Peter Gotzsche. Instead we should orient our attention to a much broader consideration of the democratic and scientific improvements needed in Cochrane. If the measures taken by Cochrane to overcome this crisis are circumscribed to Gotzsche it will be a missed opportunity for a genuine regeneration of the organization and the defense of the credibility of its work.

It is extremely superficial and probably an act of bad faith to focus the present crisis of the Cochrane Collaboration on one individual´s behaviour. Here “personal behaviour” is being used to avoid a serious debate on the future strategy and policies of the organization.  Of course, there are all kinds of people with different characters and different temperaments as in any large organization. Yes, there have been some passionate and sometimes overly heated discussions concerning important policy issues of Cochrane in which both the Cochrane leadership, including its CEO, and Peter Gotzsche have been involved. But this crisis is not about style but substance.

One person´s personality can sometimes be bothersome to some people but, without a doubt, what has moved the Cochrane leadership to take the exceptional decision to expel Peter Gotzsche are his very visible actions in the fields of science, policy and medical ethics. Peter´s positions on unethical practices by the pharmaceutical industry, the harms and overuse of psychiatric drugs, deadly secondary effects of many medicines, the dangers of over-diagnosis of mammography screening, the general inefficacy of influenza vaccines, the concealment of clinical trial data, ADHD, HPV vaccine reviews, EMAs transparency policies and medicine patents, among other issues, have many times caused great discomfort to Cochrane´s leadership. Peter Gotzsche never claimed to represent the whole Cochrane organization (only his Nordic Cochrane Centre) but his great public exposure made his Cochrane affiliation at the Nordic Centre very inconvenient for the Cochrane leadership and its “comfort zone brand” strategy. This deliberate confusion of affiliation with representation has been used by the Cochrane leadership over the past few years to attack and try to erode Gotzsche´s prestige related to his scientific and policy positions. As Gotzsche´s activities generated  wider and wider public debate, the Cochrane leadership became more and more worried about being identified with the “radical” views of one of their most famous members. 

A confrontation of paradigms: a collaboration or a “brand”?

Practically all observers of Cochrane, including a number of medical journals, the press and even the so-called “independent Counsel” hired by the Cochrane leadership, have all admitted that there are confronting paradigms of varying degrees concerning what the future of the organization should look like. One stresses the top-priority of “a sustainable business model” based mainly on substantial publishing income (produced by paywalls), capable of maintaining a large central office for editorial and administrative teams. This paradigm held by the CEO and a small majority of the Governing Board deems that the preservation of a unified “brand” and a more centralized and authoritarian “corporate image” is of the upmost importance for the financial growth and stability of the Cochrane central office. In this context the scientific, financial and policy independence of Cochrane centres outside of the UK could pose a threat to the consolidation of this common “brand”. According to this narrative held by the Cochrane leadership all other issues of policy, scientific methodology and ethics are secondary, or are even  considered “negative liabilities”, for the maintenance of the central organization. Hence, with the same logic, the present executive team and Governing Board presidency are openly reticent of contact with most public health NGOs and against any visible Cochrane leaders taking clear public positions on transparency, open data, open science or medical innovation policies.

The other paradigm, often held by a number of Cochrane “old-timers” as well as young newcomers, stresses, in varying degrees(totally irrespective of their positive or negative appraisal of Peter Gotzsche), support for much stronger policies to avoid  biases and conflicts of interest in Cochrane reviews, much greater visibility of Cochrane in policy debates on health technology evaluation, open access publishing, shared structured data and open models of biomedical innovation. What is essential for this group is where the “evidence” comes from, who pays for it and if all the clinical evidence is publicly available or not. As well, this group values much more than the present Cochrane leadership interactions with civil society organizations, NGOs and progressive policy-makers. Here a more horizontal governance of the organization is often requested with much more input from Cochrane´s regional centres which often feel marginalized from decision making processes. The present Cochrane leadership has become generally conservative, reactive to change and principally driven by its scientific publishing economic interests more than public health concerns. 

If we consider the expulsion of Gotzsche and the recent strategic direction of Cochrane from a structural social-economic perspective, the big winner in this conflict has been the pharmaceutical industry, having succeeded in weakening the voice of one of its greatest critics and having consolidated a Cochrane leadership closer to industrial interests with fewer audible critical voices. As far as we know there is no smoking gun of direct industry influence (aside from the permitted conflict of interest of reviewers), but from a cold analytical viewpoint the objective outcome is clear.

Democratic deficits and a Governing Board that does not govern

Any open, internal debate within Cochrane concerning Cochrane policies is considered by the present leadership as dangerous. This is reflected in the lack of democratic efficacy of many of the internal structures from the Council to the Governing Board, from the Annual General Meeting to the Meetings of centre directors. None of these structures have proven satisfactory forums for a fruitful relationship between the Central Executive Team and the Cochrane members that carry out most of the organization´s work around the world. The Cochrane leadership has often shown disdain and impatience with any criticism of their work or proposals. This has created a negative environment for the positive synergy among Governing Board members, members of the Council and many centre directors.

What is specially grievous is that the Governing Board, as a whole, does not govern. It seems to be considered by the central executive team to be a mere rubber-stamp for their decisions. Only the co-chairs of the Governing Board seem to have some fluid input into the decision-making process.

Every six months there is a Governing Board meeting. A few days before each meeting the members are sent dozens of pages of documents of proposals to be voted in the meeting. The response can be “yes, no or abstention” while significant amendments to the proposals are practically impossible. In other words, important decisions are taken on a “take it or leave it” basis. The highly polished proposals presented by the central executive team are not prepared in collaboration with most members of the Governing Board or other important members of the Collaboration, in what could be an enriching inclusive process between Board meetings. Instead, most members of the Board are presented with a series of fait acompli which has created quite a deal of frustration among veteran members of Cochrane on the Board. One of them said that he/she felt like resigning at every board meeting due to being taken for granted by the CEO and the central executive team. Serious discussion of strategic policy, scientific and organizational issues is not common in Governing Board meetings and when it does occur it is not well received by the Cochrane leadership, including the two co-chairs of the Governing Board who never have a public word of discrepancy with anything presented to the Board by the CEO. Despite vocal criticism from veteran Cochrane members concerning the weak role of the Governing Board compared with the decisive role of the central executive team, no measures have been taken to improve the democratic dynamics of the organization. A great deal of the time of Governing Board meetings is occupied by long power-point presentations given by the central executive team about their accomplishments.

There is a general lack of democratic participation and debate among the members in Cochrane. For example, the Strategy 2020 adopted by the board has never been broadly discussed debated among Cochrane members. Another illustrative example of this top-down control obsession of the Cochrane leadership is the “webinar” organized a few days ago to theoretically “explain” the current crisis sparked by the expulsion of Peter Gotzsche (though no convincing evidence was provided). All the microphones of the participants were muted who they were not allowed to speak, only to listen passively, and even their written questions sent to the CEO and the Co-Chairs were “re-interpreted” and formulated in different terms. One Cochrane centre director participating in the “webinar” asked in his written question «Would you be ready to offer yourselves for a motion of confidence, that is, resigning from the Governing Board and also applying as candidates to the new elections? This would give Cochrane people the opportunity to explicitly support you.” The question announced to the whole group by the organizers with no respect for the written formulation was: «He asks if the Governing Board should resign». This is just an anecdotal example of the generally top-down, fearful approach to democratic debate and participation in Cochrane.

The anti-democratic crucifixion of Peter Gotzsche

The whole process against Gotzsche has been anti-democratic and none of the basic tenets of due process, fairness and transparency have been upheld. There has been no attempt at seeking outside neutral arbitration nor the use of techniques of conflict resolution with the aim of reaching a friendly agreement or even a more peaceful modus vivendi concerning disagreements. No time has been given to establish an independent committee of conflict made up of people from outside of Cochrane´s the main institutions. Every step of democratic guarantees that is common in most large organizations has been ignored in this case with the objective of the rapid exclusion and tarnishing of Peter Gotzsche. Any future independent investigation of this question should be centered on the violation of democratic processes by the CEO and the Co-chairs of the Governing Board.

Instead of an open and balanced procedure with the hope of reaching a friendly arrangement or a fair arbitration, the whole process was practically limited to a totally unproductive written back and forth between Cochrane´s CEO and Peter Gotzsche.

Peter Gotzsche has had no chance to defend himself in person before the Board. He was expelled without even knowing what he was accused of, aside from the generic accusation of causing “disrepute” to Cochrane. The only accusation that had been made clearly, that he had violated the spokesperson policy, was not at all confirmed by the so-called “independent counsel” that found the spokesperson policy “ambiguous” and “open to different interpretations”. In fact, the so-called “independent counsel” did not reach any clear conclusions nor did it make any recommendations for disciplinary action against Gotzsche despite this being requested from the Cochrane leadership. The ambiguous phrase used publicly by the Cochrane leadership that the open-ended counsel report  – “did not exonerate”- falsely insinuated that Gotzsche was found guilty of wrongdoing on the part of the Counsel, but that did not happen. It is disgraceful that the Cochrane leadership has used such personal defamation tactics without any proof nor transparency.

The decision to expel Peter Gotzsche from membership in Cochrane by a minority of the Board was taken by such a narrow margin (6 in favour and 5 against with one abstention – without the presence or vote of Gotzsche) that any rational consideration of an issue that divides the organization would have called for a postponement, a reconsideration or a new approach of conflict resolution. To move forward with this unprecedented decision “whatever the cost” was a very reckless course to take, to say the least.

What is totally unacceptable and probably illegal is that dark and ominous insinuations have been made about Peter Gotzsche, backed up with absolutely no evidence. Concerning his personal “behaviour”, the Cochrane leadership has publicly and privately used  the language of the “me-too” movement and “zero-tolerance” of sexual harassment and abuse. 

The  exclusion of Peter Gotzsche and the “suggested” resignation of another four members of the  Board was a well-planned, pre-determined operation for the elimination of all the critical voices from the Governing Board. Shortly before the Governing Board vote that expelled Gotzsche, one of the six members of the Board that voted in favour of the expulsion, stated that all the members of the Board were obliged to publicly defend the decision and not reveal the details of the close vote that was about to take place. What was also sought by the Cochrane leadership was a concealment of what had happened in the process, debate and vote.

Should up to half of Cochrane authors have conflicts of interest?

At the Governing Board meeting in September 2017 Peter Gotzsche proposed a text, with the support of a number of other members of the Board, to substantially strengthen Cochrane´s conflict of interest policy which today allows up to half of the authors of reviews to have conflicts of interests with the company that makes the product they are evaluating. This proposal was met with considerable resistance and outright discomfort from the Cochrane leadership, one of the Governing Board leaders even said that “without conflicted reviewers we´ll find no-one to do our reviews”. Over the next year no progress was made on this conflict of interest proposal and a long, torturous bureaucratically procedure was suggested by the co-chairs with the intention of burying the whole issue.

Long time Cochrane collaborator Ray Moynihan has insisted that an important improvement of Cochrane´s conflict of interest policy is long over due because it still allows individuals with financial ties to pharmaceutical companies to review evidence about those same companies’ product if they make up less than half of the review team. Moynihan states “it’s an anathema that conflicted individuals should be reviewing what is often conflicted evidence to start with. Cochrane has an opportunity to provide global leadership by cleaning up this mess”. Unfortunately, the Cochrane leadership has shown no willingness to clean up the mess.

The International Society of Drug Bulletins has stated it very clearly: “What is at stake is the not the transparency of conflicts of interest or whether or not it is feasible to get rid of conflicts of interest; it is definitely about trust, credibility and scientific integrity. Cochrane is damaging the trust and credibility that doctors, pharmacists, scientists and patients have put in them. Cochrane’s credibility and trust are largely at stake if they do not adequately deal with this issue immediately.”

It is no coincidence that Peter Gotzsche´s expulsion took place when he had been insisting for over a year on a new, much stricter conflict of interest policy for Cochrane.

The lack of commitment to advocate for transparency, better evidence, public-interest health technology assessment and open innovation models

The present Cochrane leadership represented by its CEO is very reticent and even allergic to any public interest advocacy despite that it is one of the important elements of the Cochrane 2020 strategy. In fact, the internal auditing of the degree of fulfillment of the established advocacy objectives of the 2020 admits that Cochrane´s public advocacy has been totally insufficient and is marked as a failure with the colour “red”.

As a member of the Board I took the initiative to make a number of concrete suggestions for Cochrane public advocacy by means of mails, phone calls and personal meetings. I requested that Cochrane take positions on the weak application of clinical trial transparency rules in the EU, on the new health technology assessment legal framework of the EU and on public interest innovation proposals at the WHO.

Over the past year the CEO has insisted that they “are not ready” for taking public positions, that it is not a present priority for the staff and expressed in a written reply the need to plan advocacy carefully based on the “products” (systematic reviews) Cochrane develops. When one top member of the Cochrane team was asked about Cochrane´s relationship with the major public health NGOs that often present proposals for access to medicines and new open innovation models before the World Health Organization, he/she said that these organizations held viewpoints “too radical” for Cochrane.

At the Edinburgh meeting in September, 2018 the central executive team presented an advocacy proposal for 2019-2020 without structuring any previous input or dialogue from members of the Governing Board despite the continuous interest in advocacy of a number of members of the Board.

Despite visiting the London office on my own initiative, speaking with members of the executive team and supplying political intelligence in writing, I received no corresponding requests to collaborate, offer ideas or comment on the advocacy strategy or other related issues before it was presented for the consideration and approval by the Board.

What is evident to any observer is that over the past few years Cochrane has not considered it important to influence public policy in areas extremely relevant and necessary for the production of “better evidence”.  Apparently, there is a divergent viewpoint over what kind of public advocacy, if any, is based on the needs of its “products”.

Is Cochrane allowing evidence-based medicine to be “hijacked”?

While Cochrane has been considered by many to be the beacon and the best example of “evidence based medicine”, the Cochrane leadership has generally not heeded very qualified and documented calls concerning the perversion and hijacking of EBM.

As John Ioannidis has said ““evidence-based medicine” has become a very common term that is misused and abused by eminence-based experts and conflicted stakeholders who want to support their views and their products, without caring much about the integrity, transparency, and unbiasedness of science.” Some observers feel that many Cochrane reviews are being “misused and abused” in this very manner by the pharmaceutical industry. Especially criticized is the production of many Cochrane reviews based on journal articles without attention given to much of the clinical data which is often either hidden, censored or manipulated by the industry sponsors of the trials. In these journal-based reviews there is often insufficient importance given to the factors of publication bias, the concealment of secondary effects and changes in statistical protocols.

Can a systematic reviews of journal articles, often suffering from publication bias and the lack of accessible raw data or structured data to back them up, supply the “trusted evidence” Cochrane promises? According to Cochrane veteran Tom Jefferson the answer is: “probably not”. Most journal articles have a very high degree of bias that usually exaggerate benefits and hide possible harms. Moreover, many journal articles are based on clinical data that is not available or hidden by the trial sponsors. Because of these reasons Jefferson considers much of the raw material used in Cochrane systematic reviews as “garbage”. Jefferson suggests alternatives to focussing on unreliable journal articles: “They (Journal Articles) can be carefully contrived pieces of marketing, part of a global jigsaw. We can only guess at what their purpose is and what the true results are. We need to stop producing reviews based on articles (or at least solely on articles) and seriously and urgently look at drawing from data sources which allow alternative explanations and conclusions from the data, because the data set is detailed and near-complete.”

The present Cochrane leadership has not reacted proactively to this criticism of how many of Cochrane´s systematic reviews are carried out within a biased and nontransparent context due to the manner a large part of biomedical evidence is generated, evaluated and published. They have even refused to seriously consider other emerging forms of evidence synthesis such as realist reviews, scoping reviews and some kinds of rapid reviews which could be very important for decision makers.

Cochrane has also not been positively responsive to public criticism by many longtime members and supporters of Cochrane that Cochrane reviews often do not include open access to structured data for sharing, re-use and to back up conclusions*. While Cochrane formally supports the clinical data transparency initiative All Trials “it has no similar clear principles on opening full access to the data within Cochrane reviews”.

There has been a positive disposition for dialogue on these issues on the part of Cochrane´s chief editor David Tovey but in his response he insists that a “more liberal application of open access” would jeopardize Cochrane´s “financial sustainability”. Again we see an unresolved conflict between the needs of the prevailing Cochrane business model and its declared public interest objectives of openness and the sharing of data.

Without taking energetic steps to improve the independence, openness,  transparency and reliability of the evidence it uses, the credibility of Cochrane reviews will decrease.

David Sackett´s definition of evidence-based medicine is “integrating individual clinical expertise with the best external evidence”. There are ample examples that this goal has not generally been achieved for a number of reasons. One major reason is what Ioannidis has insisted that clinical evidence is “becoming an industry advertisement tool” and that “much ‘basic’ science [is] becoming an annex to Las Vegas casinos due to a highly competitive, unpredictable mass of aggressive gamblers with enormous economic stakes in play.” It is has become evident that the present Cochrane leadership has sometimes chosen to ignore overt industry manipulation of clinical evidence and has ocassionally even fallen in the trap of serving as “an industry advertisement tool” with a shiny Cochrane stamp on it that lends this publicity “independent” credibility.

Peter Gotzsche and others in Cochrane have defended the idea that evidence generated by companies with a vested financial interest in the marketing of the “reliability” of that evidence is a great problem for medical researchers and the carrying out of systematic reviews. Most of the Cochrane leadership thinks and acts otherwise in the way it treats the evidence usually used as the “raw material” for systematic reviews. If that is added to the fact of a weak conflict of interest policy that allows up to half of reviewers to have conflicts of interest, Cochrane has a growing credibility problem.

The objective of some Cochrane authors is to produce as many reviews (referred to as “products” by the Cochrane CEO) as possible, often by networked meta-analysis, which according to Cochrane´s own handbooks has significant limitations and weaknesses. But when the principal objective is to turn out a large quantity of reviews by means of collecting the maximum quantity of “evidence”, without much scrutiny of its origin, the protection of patients, the defense of public health and the rational use of medicines can become of secondary importance.

The regeneration of Cochrane: how to come out from the crisis stronger

In order to overcome the crisis in Cochrane and to defend the prestige of the organization it is crucial to re-establish the the conditions of trust and cohesion necessary to strengthen Cochrane in its work and goals. The Ibero-American centre directors have made a  proposal that calls for the election of the posts vacant on the Governing Board that would name an independent investigative commission to investigate the process of the expulsion of Peter Gotzsche.

For many reasons this would be a false closure of the crisis because it leaves out many of the fundamental underlying aspects of this crisis. To exclusively focus the solution to the problems of Cochrane on the personal case of Peter Gotzsche´s “behaviour” while ignoring at the same time the crucial democratic deficits and strategic differences that have led to the conflict, would produce a damaging missed opportunity for the democratic regeneration of the organization and the improvement of its scientific work.

The conditions for the resolution of the conflict must have minimum guarantees which are consistent with democratic demands of impartiality and objectivity, something that cannot occur if the organization of the investigation process remains in  the same hands of the same Cochrane leadership that were active participants in the decisions that led to the expulsion of Peter Gotzsche. The people who have actively participated in this conflict, principally the CEO and the co-presidents of the Governing Board, are not credible organizers of  a fair and transparent electoral process of new members of the Board of government and much less the establishing of the terms of reference and objectives of an investigative commission.

Below are some of the reasons why this proposal would imply a false closing and at the same time  an alternative proposal is made  for the broader regeneration of the organization.

(1) The small number of people currently members of the Governing Board, who have taken part and adopted decisions in the conflict, must validate and receive the democratic support of the whole organization if they aspire to continue as members of the Board. Consequently, it is neither sensible nor appropriate that elections be held to only fill the vacant positions of the Governing Board, when the coherent and logical in these exceptional circumstances would be the complete renewal of the Governing Board. 

(2) Any electoral process for the election of positions of responsibility and direction must have conditions of neutrality, objectivity and monitoring. For this reason, an independent commission must be created specifically in charge of guaranteeing the election process. The members of an electoral commission in charge of supervising the election process must enjoy maximum independence and must be elected by a joint meeting of the Council, the Governing Board and the center directors. The electoral commission for the electoral process must assure a public and transparent census of the members of Cochrane. The legitimacy and validity of the voting process and candidacies must be established from the census of Cochrane members existing prior to the current crisis that started in the month June 2018.  The electoral commission created must establish sufficient time periods and open forums of debate to provide for a democratic campaign and debate. 

(3) Any proposal to create a commission of inquiry that is appointed and oriented in its tasks by the current leadership of Cochrane, including the remaining  Governing Board(5 out of 13) or the executive team, does not offer guarantees of impartiality. Moreover, this inquiry commission should integrate broader objectives beyond the reasons Gotzsche expulsion and also consider the organizational, democratic and strategic issues that are substantially involved in the current conflict.  

(4) An investigation of the conflict focusing exclusively on the process of expulsion of Peter Gotzsche would in fact exclude the necessary evaluation of the democratic deficits of the functioning of the Cochrane governance bodies, those related to the conflict of interest policy of Cochrane reviewers and the consequences of the publishing business model of the organization. 

(5) The  selection of a new editor-in-chief of Cochrane to replace David Tovey is a decision of vital importance for the present and future of the organization. This requires a process with broad democratic foundations that cannot be left in the exclusive hands of the CEO and the co-presidents of the Board of Governors who, in turn, have been active protagonists of the current crisis.

David Hammerstein, ex-member of the Cochrane Governing Board, resigned September 13^th, 2018

Annex: Analysis and context of the scientific conflict in Cochrane

.