When Is Science Valid?

For scientists, peer review is the time-tested method for establishing scientific facts. But how can we educate a public that is growing accustomed to challenging a consensus they don’t understand?

OPN covers

Science has evolved over thousands of years of human inquiry to provide a rational basis for understanding and predicting what happens in the world around us. We rely on science to enhance our standard of living, to keep us healthy, and to address the problems and challenges that we face as a society and a species. Scientists have—by and large—demonstrated the overall robustness of the scientific quality-assurance process by developing a peer-review system and implementing mechanisms for addressing plagiarism, fraud and other human failings that compromise results.

Yet more and more, scientific information is being challenged by politicians, vested interest groups, the media and the public. Nowhere is this more apparent than in the attacks made recently on the methodology used by the Intergovernmental Panel on Climate Change. As this example makes clear, human idiosyncrasies will always contribute a (hopefully small) degree of error amongst a large body of truth.

In an era characterized by the open transmission of information—and one in which sensationalism propagates rapidly—how can we convey the overall robustness of science to those who have no knowledge of how science operates?

It was in response to this challenge that the Federation of Australian Scientific and Technological Societies (FASTS)—the primary advocacy group for science in Australia—published the paper When Is Science Valid? A Short Guide to How Science Works and When to Believe it. As then-FASTS president and the main author of this document, I wanted to give politicians, opinion leaders, the media and the general public a plain-English explanation of how science stays honest. The paper can be downloaded from the FASTS website.

Of course, often when scientists are faced with such an exercise, they add caveats that explain exceptions and help them to cope with the minutiae. However, as experts in science communications have well-established, there is no place for such embroidery when one is trying to convey a simple message to a non-expert audience.

The simple message is this: By and large (i.e., no caveats), science over the centuries has honed a methodology accepted by society that maximizes the reliability of the information it produces. Peer review is the robust process that we have for ensuring the validity of scientific information.

Understandably, the public and the media are often not aware of how peer review works. For example, many often equate writing a book with expertise in a field. Certainly, publishing a book is one way of disseminating ideas. However, it’s not as rigorous a test of validity as exposing an idea to scientific scrutiny in an international peer-reviewed journal. Presenting previously peer-reviewed research in a book is legitimate; however, unreviewed material published in books or articles doesn’t pass the validity test.

So, to establish whether a scientific idea has been validated, we provide a quick checklist for non-experts in the wider community to apply:

  • Has the idea been published in the peer-reviewed literature in the relevant field of science?

  • Have other scientists cited that publication as being valid (as opposed to citing it to show that it is wrong)?

  • Have other scientists conducted additional tests that show the idea to be valid?

  • Has the idea been built upon to create new understanding—in other words, has the idea become useful to other scientists or in real-world applications?

Many in the media think that an adversarial debate is a valid way of discussing scientific issues—even when the concepts they question have already been well-established by the scientific community. They do not understand that scientific consensus, as determined through peer review, may already overwhelmingly support one side of the debate. That may seem obvious when one considers a debate that has been settled for centuries—such as whether the earth is flat or that the sun is located at the center of our solar system. But very often this acceptance is ignored when it comes to modern-day issues such as human-induced climate change, despite the fact that scientists may have already reached consensus on the issue.

Of course there are some scientists who do not support the view of the mainstream scientific majority on human-induced climate change, but they represent a minority. Until that minority publishes their ideas—thereby confronting the mainstream view in the scientific literature so that the alternative views can be rigorously tested against the evidence—the wider community will continue to adopt the current climate change concept.

When it comes to politics, policymakers bear an even greater responsibility to understand the process of scientific consensus, since our representatives (in most societies) are elected to make decisions in the public interest based on the best information available to them. This is often referred to as evidence-based policy. Politicians thereby have an obligation to understand where the scientific consensus lies in order to make informed decisions, and to weigh this against wider social, economic and political factors.

In order for science to be adopted effectively in society, all of those who participate in its creation, dissemination and application bear certain responsibilities. There are obligations:

  • On scientists, to test their ideas by peer review

  • On journalists, to present the consensus scientific view, and

  • On politicians, to accept the consensus scientific view and balance it against wider social, economic and political considerations. If a politician doesn’t accept the consensus scientific view and acts on faith, he or she should make that known, noting that they hold a different view on evolution, human-induced climate change or whatever mainstream scientific concept they are challenging.

Scientific societies such as OSA also have an obligation to ensure that the wider community is aware of the robust nature of its peer-review processes.

In order to know when the scientific evidence represents a true consensus, rather than simply being influenced by the loudest or most persistent voice, we must educate the wider community on how the scientific process works.

And that is the aim of When Is Science Valid?—to provide the understanding needed to establish whether information has been tested to the best of our ability by rigorous scientific processes, and therefore whether it can be trusted.

Ken Baldwin is a former OSA director-at-large and a member of the OSA Public Policy committee.

References and Resources

>> The Federation of Australian Scientific and Technical Societies: www.fasts.org: When Is Science Valid?—A Short Guide to How Science Works and When to Believe it.
>> Sense About Science. A similar exercise has been undertaken by the British organization Sense About Science, an independent charitable trust that responds to misrepresentation of science and scientific evidence.

Got Views? If you are interested in sharing your perspective on an issue relevant to science, optics or science policy in OPN’s Viewpoint column, please contact us at opn@osa.org.


Publish Date:

When Is Science Valid?

For scientists, peer review is the time-tested method for establishing scientific facts. But how can we educate a public that is growing accustomed to challenging a consensus they don’t understand?

Become a member or log in to view the full text of this article.

OSA Members get the full text of Optics & Photonics News, plus a variety of other member benefits.

Publish Date:

Add a Comment