Thoughts on the ocean, the environment, the universe and everything from nearly a mile high.

Panorama of The Grand Tetons From the top of Table Mountain, Wyoming © Alan Holyoak, 2011

Tuesday, February 21, 2012

On Science 2: The Limitations of Science

The scientific approach is a powerful way to discover natural laws and describe how the universe works, but it is not all-powerful.  Science has limitations that any student of the sciences needs to be aware of.

Limitation #1: Science is objective and empirical

The scientific approach can be used to address only questions that are objective and empirical.  An objective question is one for which a definitive answer actually exists.  And a question is empirical only if the answer can be discovered through the collection and analysis of empirical evidence = observations made via the physical senses or technology that extends the senses.

An example of an objective, empirical question is: "What is the fastest way to get to work?"  This question has an actual answer, and it can be discovered by collecting empirical observations on the amount of time it takes to walk, bike, drive, take a train, bus, helicopter, plane, etc., from point A to point B under a wide range of conditions.

An example of a question that is neither objective nor empirical is "What is the best way to get to work?"  This question can be answered any number of different ways - fastest, most economical, most comfortable, most prestigious, most environmentally-minded, etc., depending on the opinion or perspective of the question asker.  So the answer is based completely on the subjective measure a person may want to apply to it, and is therefore not scientific, because someone else is likely to apply a different subjective measure, etc.

An example of a question that is objective but not empirical is "Does God exist?"  This question is objective because there are only two possible answers to the question, "Yes" and "No".  But it is a non-empirical question since we have no way to collect empirical, demonstrable, repeatable observations that can be shown to anyone who wants to see them.  This means that science simply cannot answer questions about things outside of our ability to observe and measure the physical/natural world.  Consequently, science is God-neutral and religion-neutral.  So if anyone ever tries to content one way or another about the existence of God using only scientific evidence, you should realize that they don't understand the limitations of science, and a little warning flag should pop up in your head.

Limitation #2: Scientific explanations always contain an unavoidable element of uncertainty

One of the first things a scientist does when s/he prepares to do some research is to state what they think they will observe and explain what those observations might mean. This preliminary explanation is called a hypothesis, a Research Hypothesis to be more precise.  It would be bad process, though for a scientist to set out to prove their Research Hypothesis correct, so in order to reduce personal bias the scientist develops a second option or explanation that is the opposite or negation of their Research Hypothesis.  This second explanation is called the Null Hypothesis.

After the researcher has collected as many empirical observations as possible, given time and other constraints, s/he analyzes those data.  This usually involves the application of statistical methods.  The interesting thing about this overall process is that the Research Hypothesis should never be the one that is tested.  Rather, the Null Hypothesis is the possible explanation that the researcher has to decide to reject or fail to reject (i.e., accept).  The really interesting thing about this process is that the researcher is forced to accept the Null Hypothesis unless the data and analysis of the data provide overwhelming evidence that the Null Hypothesis is not correct.  How overwhelming?  The scientific standard for most disciplines is that the research has to be at least 95% confident that the Null Hypothesis is not correct before it can be rejected.  Of course, some data sets allow the researcher to be more than 95% confident.  Sometimes they can be 99% confident, 99.9% confident, or even more, but no matter how confident a scientist is in their decision, they can never be 100% confident.  The 5%, 1%, or 0.1% represents the chance of making a wrong decision regarding the Null Hypothesis - that it should have been rejected when it was accepted, or that it should have been accepted when it was rejected.  This unavoidable element of uncertainly means that the researcher can never be absolutely 100% confident that their decision about the Null Hypothesis is the right one, though levels of confidence typically exceed reasonable doubt.  I mean, I would LOVE it if I could be at least 95% confident that was I making the right choice whenever I make any kind of decision, wouldn't you?

You should be aware that there are people out there that use this unavoidable, but usually minuscule amount of uncertainty to say that because there is some uncertainty in the data that we should feel free to reject any scientific explanation or conclusion that we don't happen to like.  This is faulty logic, so don't be misled by the kind of misguided decision-making promoted by this faulty logic.

Limit #3: It is possible to make the wrong decision regarding the Null Hypothesis, even if the scientist applies good process

As mentioned above in Limitation #2, it is possible that the decision to reject the Null rather than accept it is not the right one, even though the data suggest that this is the correct thing to do.  It is possible, for instance, that the set of observations you collected are not really representative of a population or a real cause and effect relationship, and this can result in faulty decision-making.

Does this happen?  Yes, occasionally, and more often when a scientist has only a small set of data than when they have a large one.

Is there a way to safeguard against or reduce the rick of making the wrong decision?  Yes.  The first line of defense is called the peer-review process.  Before a scientist's work can be accepted for publication in a professional journal, their work is sent out to a hand full of experts in the field.  These experts provide a no-holds-barred critical review of the research that was done.  They critique the overall research design, amount and type of data collected, the statistical and other analytical methods employed, and the conclusions reached by the researcher.  This peer-review process catches most errors that exist in research papers before they are published.  Then, once some research is published, other researchers will read it and some of them will carry out independent tests of their own to see if the obtain similar data and conclusions.  If they do, then the conclusions of the original research are supported.  If not, they are refuted, and the other researchers publish their work after peer-review.  This process of independent replication catches most of the other errors, but even so we will never be 100% confident of our decisions, though our confidence in the decision typically exceeds any reasonable person's criteria for surpassing reasonable doubt.


So to sum up, there are three primary limitations of science:
  1. Science can address only objective, empirical questions.
  2. Scientific conclusions and explanations always contain at least a tiny amount of uncertainly
  3. Scientists can make the wrong decision about what to accept when they do their research
I hope that this overview has been helpful and will help you understand what the main limitations of science are, and how risks of reaching faulty decisions and explanations can be minimized.

No comments:

Post a Comment