From: Jim Tung <jimtung**At_Symbol_Here**gmail.com>
Subject: Re: [DCHAS-L] Debunking Bad COVID-19 Research
Date: Wed, 1 Jul 2020 11:18:54 -0700
Reply-To: ACS Division of Chemical Health and Safety <DCHAS-L**At_Symbol_Here**Princeton.EDU>
Message-ID: CAPvjKFH0pNh159O=eLnDOTdAdpC27ty9TAW=R-z2=HKdm4NznQ**At_Symbol_Here**mail.gmail.com
In-Reply-To


Hello, all:

I write to respond to the invocation of the Amgen story. Eight years later, Amgen has never revealed the 53 studies that it found to be problematic. While I agree that the Amgen team found a problem, to use irreproducible means to accuse other studies to be irreproducible is in itself weak stew.

Best wishes, Jim



On Tue, Jun 30, 2020 at 8:09 AM ILPI Support <info**At_Symbol_Here**ilpi.com> wrote:
Good science is reproducible. However, it's an open secret that a significant amount of biomedical (and other) research is not. Here are a couple slides from a lecture called Paradigms and Pseudoscience I do in my Nobel Prize course. Keep in mind this is discussing high-impact papers that passed peer review in some of the highest impact journals out there

Slide 1 - Good Science is Reproducible

• 2012, Nature - Amgen scientists look at 53 landmark cancer studies. Confirmed only 6 (11%).
• 2011 - Bayer Scientists found only 25% of published preclinical studies could be validated.
• These papers spawned hundreds of other secondary studies that did not seek to confirm or falsify the original work..
• Secondary research included clinical studies. Wow.
• Reproducible studies- "authors had paid close attention to controls, reagents, investigator bias and describing the complete data set."
• Others - plagued by lack of double blind control studies, presentation of a single result or data point, supplying data that supports their hypothesis but discarding data that does not!


Slide 2 - This is Widespread

• NIH official comments 75% of published biomedical findings would be hard to reproduce.
=E2=80=A2 Smaller studies more prone to false conclusions.
• Nobody gets a publication/credit for reproducing work.
• "Publish or perish" makes people push out work prematurely.
• Reviewers seldom look at supplemental material and (in chemistry) do not reproduce the experiments.

See
"Unreliable Research: Trouble At The Lab", The Economist, 2013, Oct 19th 2013
http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble

Slide 3 - The Math (using figures from the above-see graphic titled "Unlikely results")

=E2=80=A2 Assume 1,000 hypotheses, of which 100 are true.
• Assume false negatives will result in 80% (80) of them being found.
• Of 900 false hypotheses, 5% (45) will be false positives for various reasons.
• This makes 125 positive results, so (45/125) = 36% of the results are bogus!
• Negative results are 97% trustworthy.
• But no journals are interested in negative results!

Key poins - you don't get tenure, you don't get grants, and your company doesn't make money for replicating someone else=E2=80™s research. And this leads to a dangerous cascade of research and effort based on unconfirmed results.

Ask me about peer review next-.

Rob Toreki


======================================================
Safety Emporium - Lab & Safety Supplies featuring brand names
you know and trust. Visit us at http://www.SafetyEmporium.com
esales**At_Symbol_Here**safetyemporium.com or toll-free: (866) 326-5412
Fax: (856) 553-6154, PO Box 1003, Blackwood, NJ 08012




On Jun 30, 2020, at 10:46 AM, Ralph Stuart <ralph**At_Symbol_Here**rstuartcih.org> wrote:

It will be interesting to see if this approach of peer review after publication becomes a trend in the scientific publishing world as in person meetings become less common and preprints rise in prominence in all fields...

- Ralph

Begin forwarded message:

From: Robert C Michaelson <rmichael**At_Symbol_Here**northwestern.edu>
Subject: [CHMINF-L] Debunking Bad COVID-19 Research
Date: June 29, 2020 at 4:15:33 PM GMT-4

From today's Inside Higher Ed

"MIT Press and the Berkeley School of Public Health are launching a new COVID-19 journal, one that will peer review preprint articles getting a lot of attention -- elevating the good research and debunking the bad.
The Rapid Reviews: COVID-19 journal will be led by Bertozzi, who will serve as the first editor in chief. Unlike a traditional journal, authors will not submit their work for review. Instead, the Rapid Reviews team will select and review already-published preprint articles -- a publishing model known as an overlay journal."

In addition to it being an overlay journal, "Bertozzi hopes that by encouraging reviewers to attach their names to publicly published reviews, transparency and accountability will promote thoughtfulness and care. Bertozzi hopes that each preprint article selected for review will be reviewed by at least two experts in the field. The reviews will themselves be reviewed to ensure they meet certain quality benchmarks, Bertozzi said."

Bob Michaelson
retired librarian

Ralph Stuart, CIH, CCHO
Chair
American Chemical Society Committee on Chemical Safety





--- For more information about the DCHAS-L e-mail list, contact the Divisional membership chair at membership**At_Symbol_Here**dchas.org Follow us on Twitter **At_Symbol_Here**acsdchas

--- For more information about the DCHAS-L e-mail list, contact the Divisional membership chair at membership**At_Symbol_Here**dchas.org Follow us on Twitter **At_Symbol_Here**acsdchas
--- For more information about the DCHAS-L e-mail list, contact the Divisional membership chair at membership**At_Symbol_Here**dchas.org Follow us on Twitter **At_Symbol_Here**acsdchas

Previous post   |  Top of Page   |   Next post



The content of this page reflects the personal opinion(s) of the author(s) only, not the American Chemical Society, ILPI, Safety Emporium, or any other party. Use of any information on this page is at the reader's own risk. Unauthorized reproduction of these materials is prohibited. Send questions/comments about the archive to secretary@dchas.org.
The maintenance and hosting of the DCHAS-L archive is provided through the generous support of Safety Emporium.