Selective reporting of results in Abstract and Research Summary of Pfizer vaccine report [BNT162b2 mRNA Covid-19 Vaccine] [English version]
Exposición sesgada de los resultados en el “Abstract” y el “Research Summary” del estudio publicado de la vacuna Pfizer [Vacuna BNT162b2 mRNA Covid-19] [Versión española]
Introduction
Introducción
Errors are possible in science, and progress consists in identifying and correcting them. In this letter to the editor of the New England Journal of Medicine, we point out a major error in the published text about Pfizer BioNTech’s covid19 vaccine clinical trial.
The Journal has rejected the letter (because “the space available for correspondence is very limited”), and has not corrected the error. We regret both facts. That is why we have again written to the editor of the New England Journal of Medicine:
«We are sorry to hear that you will not publish our letter. Apart from our letter itself, the far more important issue is the problems we wrote about. Will the NEJM be issuing any corrections to the Polack et al. Trial publication (https://doi.org/10.1056/NEJMoa2034577)? Our letter documented how the article mixed post hoc and prespecified analyzes without making any distinction between the two (in violation of a key tenet of clinical trial interpretation) and also the mixing of results from two different time points without mentioning this».
Los errores son posibles en ciencia, y el progreso consiste en identificarlos y corregirlos. En esta carta al director del New England Journal of Medicine señalamos un error importante en el texto publicado sobre el ensayo clínico de la vacuna covid19 de Pfizer BioNTech. La revista ha rechazado la carta, y no ha corregido el error. Lamentamos ambos hechos. Por ello hemos escrito de nuevo al director del New England Journal of Medicine:
«Lamentamos saber que no publicará nuestra carta. Aparte de la carta en sí, lo importante es el problema que señalamos en la misma. ¿El NEJM publicará alguna corrección al texto de Polack et al. (https://doi.org/10.1056/NEJMoa2034577)? En nuestra carta documentamos que en el artículo se mezcla análisis post hoc y preespecificado sin hacer ninguna distinción entre ambos (violando un principio clave en la interpretación de ensayos clínicos) y también la combinación de resultados de dos diferentes tiempos sin mencionarlo».
Letter sent (December 19 2020) and rejected (December 30 2020).
New England Journal of Medicine.
Selective reporting of results in Abstract and Research Summary of Pfizer vaccine report [BNT162b2 mRNA Covid-19 Vaccine]
Words: 175 (limit: 175)
We welcome the “Research Summary” of BNT162b2 mRNA Covid-19 vaccine (NCT04368728),[1] but elements of it are not consistent with good reporting practices.
In the Research Summary and article abstract, cases of “severe Covid-19” (1 on vaccine; 9 on placebo) are reported with onset from day 1 of the trial. This should have been denoted as a post-hoc analysis. The protocol pre-specified, as a secondary endpoint, reporting severe Covid-19 cases with onset 7 days after dose 2.[2] Had this been reported, the reported result would have been 1 versus 3 (vaccine efficacy 66.4%, 95% credible interval -124.8 to 96.3).[3]
Although post-hoc, reporting severe Covid-19 cases from day 1 is arguably more meaningful to patients than endpoints that only begin accruing after weeks into a trial, as was the case for the primary endpoint (symptomatic Covid-19). However the two endpoints were displayed side by side without acknowledging that they are not comparable. If symptomatic Covid-19 was reported on the same timescale as severe Covid-19, vaccine efficacy estimate should have been reported as 82.0% (95% CI, 75.6, 86.9).[3]
Peter Doshi, PhD, associate professor, University of Maryland School of Pharmacy
Juan Gérvas, MD, PhD, retired general practitioner, Equipo CESCA, Madrid, Spain
References
[1] Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, Pérez Marc G, Moreira ED, Zerbini C, Bailey R, Swanson KA, Roychoudhury S, Koury K, Li P, Kalina WV, Cooper D, Frenck RW Jr, Hammitt LL, Türeci Ö, Nell H, Schaefer A, Ünal S, Tresnan DB, Mather S, Dormitzer PR, Şahin U, Jansen KU, Gruber WC; C4591001 Clinical Trial Group. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020 Dec 10. doi: https://10.1056/NEJMoa2034577 Epub ahead of print. PMID: 33301246.
[2] Pfizer trial protocol. https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf (retrieved Dec 18, 2020)
[3] FDA. FDA Briefing DocumentPfizer-BioNTech COVID-19 Vaccine. Dec 10, 2020. https://www.fda.gov/media/144245/download (retrieved Dec 18, 2020)
COI statement: Dr. Doshi has received travel funds from the European Respiratory Society (2012) and Uppsala Monitoring Center (2018); grants from the FDA (through University of Maryland M-CERSI; 2020), Laura and John Arnold Foundation (2017-21), American Association of Colleges of Pharmacy (2015), Patient-Centered Outcomes Research Institute (2014-16), Cochrane Methods Innovations Fund (2016-18), and UK National Institute for Health Research (2011-14); and is an editor at The BMJ and unpaid member of the Reagan-Udall Foundation for the FDA.
COI statement: Dr. Gérvas has no any COI.
Carta enviada (19 de diciembre de 2020) y rechazada (30 de diciembre de 2020). New England Journal of Medicine.
Exposición sesgada de los resultados en el “Abstract” y el “Research Summary” del estudio publicado de la vacuna Pfizer [Vacuna BNT162b2 mRNA Covid-19]
Palabras: 175 (límite: 175)
Celebramos poder leer el “Research Summary” del ensayo clínico de la vacuna Pfizer de ARNm Covid-19 BNT162b2 (NCT04368728) [1], pero algunos de los resultados expuestos no cumplen con las buenas prácticas de publicación.
En el el “Abstract” y el “Research Summary” se recogen casos de “Covid-19 grave” (1 con vacuna; 9 con placebo) desde el día 1 del ensayo lo que debería haber sido señalado como un análisis post-hoc pues el protocolo preespecificó como criterio secundario de valoración la notificación de casos graves de Covid-19 con inicio 7 días después de la segunda dosis [2].
Si se hubiera tenido en cuenta dicho criterio inicial, el resultado publicado habría sido de 1 versus 3 (eficacia de la vacuna 66,4%, intervalo de confianza 95% -124,8-96,3) [3].
Aunque sea un análisis post-hoc, es posiblemente más importante clinicamente conocer los casos graves de Covid-19 desde el día 1 pues el criterio primario (Covid-19 sintomático) se cumplió sólo después de semanas de ensayo. Sin embargo, los dos resultados se han publicado parejos, sin comentar que no son comparables. Si se se hubiera empleado el mismo criterio para Covid-19 sintomático que para Covid-19 grave la estimación de la eficacia de la vacuna sería del 82,0% (intervalo de confianza 95% -75,6, 86,9) [3].
Referencias
[1] Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, Pérez Marc G, Moreira ED, Zerbini C, Bailey R, Swanson KA, Roychoudhury S, Koury K, Li P, Kalina WV, Cooper D, Frenck RW Jr, Hammitt LL, Türeci Ö, Nell H, Schaefer A, Ünal S, Tresnan DB, Mather S, Dormitzer PR, Şahin U, Jansen KU, Gruber WC; C4591001 Clinical Trial Group. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020 Dec 10. doi: https://10.1056/NEJMoa2034577 Epub ahead of print. PMID: 33301246.
[2] Pfizer trial protocol. https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf (retrieved Dec 18, 2020)
[3] FDA. FDA Briefing DocumentPfizer-BioNTech COVID-19 Vaccine. Dec 10, 2020. https://www.fda.gov/media/144245/download (retrieved Dec 18, 2020)
Declaración de conflictos de interés: Dr. Doshi has received travel funds from the European Respiratory Society (2012) and Uppsala Monitoring Center (2018); grants from the FDA (through University of Maryland M-CERSI; 2020), Laura and John Arnold Foundation (2017-21), American Association of Colleges of Pharmacy (2015), Patient-Centered Outcomes Research Institute (2014-16), Cochrane Methods Innovations Fund (2016-18), and UK National Institute for Health Research (2011-14); and is an editor at The BMJ and unpaid member of the Reagan-Udall Foundation for the FDA.
Declaración de conflicto de interés: Dr. Gérvas has no any COI.