Title: Critical reading and thinking
Course: MSc level academic and professional (information) skills > Section: Part 2. Critical reading/thinking and scientific writing

Critical Reading/Thinking in Science: read others’ ideas, think with your own head.

When you read something you can never really be 100% sure that everything is 100% correct. Science can be flawed for several reason, for example; influence of stakeholders or lack of time for thorough peer reviews.  So even if an article gets published it doesn't necessarily mean that is is flawless. But how do we spot what is right or wrong?

Learning objectives

After this section you will be able to:

- Understand what thinking critically entails.
- Recognize two possible biases in science.

Critical thinking

According to the Oxford dictionary, critical thinking is:

"The objective analysis and evaluation of an issue in order to form a judgement."

Watch this short video about Critical Thinking in Science:

https://youtu.be/josRsaUPAEU

In this next video some of the major principles of critical thinking are presented:

https://youtu.be/WiSklIGUblo

What's in your future?

What will you be doing in the future?

Will you be a Geoscientist?

A Technical Employee?

Or an Entrepeneur?

What’s in your future? Science!

Whatever your future may be, you will need to stay competitive and know what your collegues are doing. In order to stay competitive you will need to read science. But to read science you need to understand it and be able to critically evaluate it to estabilish which information is good and which is not.

There are two main issues that can affect they quality of published science:

- Involuntary bias
- Epistemic bias

We will discuss both biases in this part of the course.

Involuntary bias

Involentary bias: Erroneous conclusions due to limited knowledge about the tools we use.

Example*:

A scientist does research on landslides. He uses a tool to analyse the data and comes to some amazing conclusions. The results get published in a well known paper and are cited by different other papers.
A couple years later, the researchers has a conversation with the builders of the tool he used for his analysis. He finds out that he missed some crucial information when using the tool for his analysis. His conclusions turn out to be all wrong.

However, in the meantime other articles are published based on the same faulty conclusions. What started with a mistake, turned into science through citation in other work.

The involuntary bias is difficult to spot, because the author's are not aware of it.

*This example happend to scientist Luigi Lombardo in his 2014 article:

Lombardo, L., Cama, M., Maerker, M. & Rotigliano, E. (2014). Test of transferability for landslides susceptibility models under extreme climatic events: application to the Messina 2009 disaster. Natural Hazards, 74, 1951–1989. https://doi.org/10.1007/s11069-014-1285-2

Involentary bias question

Take a few minutes to think about how you could spot an involentary bias in scientific work and write your answer down.

Checking for involentary bias

When you read an article and you want to use that knowledge in your own work, one of the ways to check for involentary biases is to ask an expert.

For someone that is an expert in a certain field, it does not take a lot of time to read the article with you and check for biases. When you do this you prevend yourself from drawing conclusions bases on incorrect information.

Another way is to read articles based on the original article you found. Read what others have to say about it and draw your own conclusion. This methods does take a lot of time and will not always bring a bias into the light (as seen with the example before).

Epistemic bias

Epistemic bias: Erroneous conclusions due to our expectations/assumptions.

With an epistemic bias you have a prior believe about the correct answer to your question and draw your conclusions based on that believe.

In science, patterns are often found and sometimes scientists believe they know what is going on based on those patterns without looking at the whole picture. They see only the evidence that supports their believe, without looking at evidence supporting other conclusions. Incorrect or too general conclusions are drawn.

Compared to the involuntary bias this type of bias is easier to spot, because as a reader you have a different assumption or idea and you can more critically see if the assumption of the other person makes sense or not.

Example*:

Some researchers do an analysis of earthquakes. The researchers say that after four earthquakes in four different locations, rainfall in those areas releases some landslides. But over time the landslides decrease in numbers and intensity.

In the abstract of their paper they say that this decrease means that the soil is recovering its strenght and that it is not related to e.g. the water table. However, they did not actually measure this and their paper doesn't offer any evidence to support this conclusion. The researchers simply saw the pattern that they wanted to see, but in reality there is no way that they can prove that in their paper. And even if they could prove it, they only worked on 4 cases.

So always take a critical look at your own work and see if the statements you make, make sense and are supported by your research.

*This example is based on a 2015 article:

O. Marc, N. Hovius, P. Meunier, T. Uchida, S. Hayashi; Transient changes of landslide rates after earthquakes. Geology ; 43 (10): 883–886. doi: https://doi.org/10.1130/G36961.1

Epistemic bias question

Take a few minutes to think about how you could spot an epistemic bias in scientific work and write your answer down.

Conclusion

Checking for epistemic bias

When reading an article keep in mind that the writer had prior assumptions about his research even before collecting the data. So is the conclusion solid? Or can there be an alternative explanation for the data? And does the author talk about these possible explanations?

Conclusions

- Lower tier as well as higher tier journals do not always guarantee quality of scientific content.
- Even well-known scientists make mistakes.
- Always be critical when reading an article.
- When using your own mind is not enough, ask around!

*With thanks to Luigi Lombardo (University of Twente) for (part of) the content.