From: Mary Ellen Starodub <mestarodub**At_Symbol_Here**COGECO.CA>
Subject: Re: [DCHAS-L] The Economist: Quantitative Research is Often Wrong
Date: Wed, 30 Oct 2013 13:29:16 -0400
Reply-To: DCHAS-L <DCHAS-L**At_Symbol_Here**MED.CORNELL.EDU>
Message-ID: 004301ced595$8f223470$ad669d50$**At_Symbol_Here**
In-Reply-To <785F30F4-7277-4045-9CD3-8013A52835DF**At_Symbol_Here**>

Thanks for the article in the Economist. In addition to a lack of rigour and a power analyses, the implications of the results must be assessed in the appropriate context.  For example, consider the potential public health implications in a situation where an epidemiological study finds “no risk of infectious disease and adverse health effects” and concludes “a person, product and procedure is non-infectious, non-toxic and safe”, but that an analysis of false negatives, if it had been done,  would have proved otherwise…..

Mary Ellen Starodub, M.Sc. – Principal

MEStarodub Consulting, Health and the Environment

Health Hazards and Product Safety, Regulatory and Environmental Toxicology and Microbiology,
Children’s Environmental Health, Infection Prevention and Control, Contaminated Sites,
Risk Analysis-Risk Assessment, Risk Management, and Risk Communication,
Research Knowledge & Translation, Science & Policy Development, Curriculum Development &Training


From: DCHAS-L Discussion List [mailto:dchas-l**At_Symbol_Here**MED.CORNELL.EDU] On Behalf Of Ralph B. Stuart
Sent: October-28-13 10:50 AM
Subject: [DCHAS-L] The Economist: Quantitative Research is Often Wrong


Quantitative Research is Often Wrong


The Economist has a nice analysis of the high probability of wrong quantitative results being published in academic journals. In one example, cancer researchers tried to replicate 53 published studies but could only confirm the findings from 6. In another example, pharmaceutical researchers only got the same result a quarter of the time when repeating 67 so-called "seminal" studies.

The article has a helpful visualization of the statistical outcome of 1,000 research studies under fairly reasonable assumptions: 125 of the studies would be published, containing 80 correct results and 45 wrong results. The remaining 875 studies would have a much higher accuracy rate of 97%, but because they didn't find anything interesting they would not be published. Because of this publication bias, only 64% of published results would be true, in this example, despite following research protocols that produced good accuracy among all (published, as well as non-published) studies.


Ralph Stuart, CIH

Chemical Hygiene Officer

Cornell University


Previous post   |  Top of Page   |   Next post

The content of this page reflects the personal opinion(s) of the author(s) only, not the American Chemical Society, ILPI, Safety Emporium, or any other party. Use of any information on this page is at the reader's own risk. Unauthorized reproduction of these materials is prohibited. Send questions/comments about the archive to
The maintenance and hosting of the DCHAS-L archive is provided through the generous support of Safety Emporium.