Is Research Free From Bias?
Scientific research isn't immune to bias - here are 5 common ones to watch out for.
Haven’t we all come across headlines like these?
“New study suggests dark chocolate reduces blood pressure”
and
“Weight loss pill effective in controlling hypertension”
Here’s the thing though. While these headlines promise us breakthroughs backed by research, they might be not as unbiased as they seem.
You may now wonder. How can research be biased? The reality is, that even the most well-intentioned research study can be influenced by conscious or non-conscious bias which consequently affects its findings.
This does not mean that all the research out there is wrong. Instead, it means that we as readers need to be more critical of research and make informed choices based on the evidence presented.
With that in mind, through this article, we wanted to introduce you to a few research biases and also how you can watch out for them. But before we proceed, let us first understand why it is important to know about bias in research.
Shortcuts For The Busy Reader
Why Should We Know About Bias In Research?
We need to be aware of bias in research because of the following reasons:
Bias is present in all studies, regardless of the study's methodology, and it isn't easy to eradicate.
Bias can emerge at any point in the study process.
Biases can impact the study's reliability and validity, leading to misinterpretation of the data.
The aforementioned reasons highlight how pervasive bias is in the context of research. They highlight the necessity for us to increase our awareness of biases to make informed choices. Whether it be the efficacy of a new drug, interpreting a news piece on a scientific study or the implementation of a new policy based on research, understanding the underlying bias only empowers you to analyse information and make sound judgements.
What Are The Types of Research Bias?
Several biases impact the research process and through this article, we are going to closely examine five types of research bias. There are:
1. Selection Bias
Selection bias is a term describing circumstances where certain groups are unintentionally excluded from research, which can lead to misleading conclusions and make it difficult to apply findings to a larger population.
Example: An ongoing weight-loss pull trial solely focuses on regular gym goers and excludes individuals with existing health conditions, dietary restrictions, or limited access to fitness centres and gyms. The sample is not representative of the pill’s target audience, i.e., those who are less active. |
More often than not, the pill’s effectiveness will be overestimated, but the lifestyle differences within the study’s sample and the target audience will rarely be considered by readers. This discrepancy between the two groups is what hinders the generalisability and real-world applicability of the study’s findings.
Selection bias can manifest in several ways, which include:
Sampling bias: when the study's sample does not represent the population being investigated. That is, sampling errors that lead to over or under-representation of specific groups in a sample.
Self-selection bias: it occurs when those who volunteer to take part in your study do not represent the target audience. For example, an online survey
imagine conducting an online survey asking people about their workplace stresses, with your participants consisting of university students.
Attrition Bias: occurs when participants discontinue or drop out of a study while it is being conducted.
Non-response bias: when people do not respond to participate in a survey or research owing to lack of time or stigma associated with the topic.
To be more aware of selection bias, we recommend carefully reviewing the methodology section of every study. This section details the participant selection process, sample demographics, and research design. Some points to consider when examining can be whether participants were chosen randomly or if they volunteered; was the sample large enough; were any groups excluded from the study, and why were they?
By evaluating the methodology, you will be able to ascertain the strengths and weaknesses of a study and draw your conclusions regarding its findings.
2. Response Bias
Response bias is a term used to signify the various tendencies of participants to provide inaccurate or false answers to self-reported questions. These forms of questions are mostly used in studies that use structured interviews or even surveys.
A reason why this bias occurs is because participants tend to resort to external cues like social norms or provide answers to please the researcher, which consequently can skew results.
Example: You take part in an anonymous survey looking at social media usage. The questions ask you the amount of time you spend on each social media app. Your consumption of social media is quite high, and to avoid being judged, you provide an inaccurate response. Despite the survey being anonymous, you end up providing a biased answer. |
There are several ways response bias can manifest itself, which include:
Agreeing to statements when you are presented with binary options i.e. “Agree/Disagree”, “True/False” and “Yes/No” (acquiescence bias).
Participants either received cues or predicted the aim of the study and adjusted their responses to align with that (demand characteristics).
Provide answers that align with societal norms and not the truth to appear favourable (social desirability bias).
The order in which questions are presented in a survey or interview can influence how participants answer subsequent questions (question order bias).
Consistently choose extreme responses on a scale (e.g., highest, lowest), even if it might not reflect the participant’s true experience or opinion (extreme responding).
As a reader, we recommend looking at the study design whenever you read a research paper. If the study is a survey or an interview, know that it can be subject to response bias.
3. Researcher Bias
Researcher bias also known as experimenter bias occurs when a researcher’s beliefs, experiences, and expectations, consciously or unconsciously, influence the research. Investigators can consciously influence a study by creating research questions or methodologies that align with their pre-existing beliefs. On the other hand, biases, stereotypes, and cultural norms can unconsciously impact a study.
Example: Imagine your teacher is conducting a study to evaluate the effectiveness of a new method tailored to improve students' comprehension. She is, however, of the belief that the older and traditional methods work better than the newer ones. This bias can influence her in several ways including
|
In the above example, we can learn how a researcher consciously or subconsciously influences the methodology of the research due to their inherent beliefs in the subject area. In addition to the above, their influence can also influence how the data will be interpreted.
For instance, if the study yields ambiguous results, they are likely to attribute the student's performance to traditional methods instead of other methods like their personal lives and external pressures which could have played a role.
These factors not only skew the results of the study but also compromise its validity and reliability. Not just that they erode the trust that the public has towards investigators and research institutions.
Trying to find researcher bias in a study can be quite tricky. But as a reader, keep an eye out for the research design and examine if there are any methodological flaws. In addition to that you can also examine if the researchers are transparent about the limitations of their study and offer alternate explanations for their findings. Lastly, you can compare two studies on the same topic to see if there is any similarity in the findings or if they are too good to be true.
4. Publication Bias
Publication bias is a form of reporting bias resulting in the selective publication of articles based on the research finding’s direction (i.e., positive or negative) and strength (how strong the finding is). As a result of this bias, studies that do not yield statistically significant results are less likely to be published.
Example: A new drug is being tested for its efficacy in reducing the symptoms of a certain disease. Study A: Shows that the drug can significantly reduce the symptoms by 30% Study B: suggests that there is no significant effect of the drug. Study C: Findings show that it has a positive yet non-significant effect, i.e. it reduces symptoms by 5% In this scenario, Study A is more likely to get published, while Study B will not see the light of day, and Study C will be published but won't gather much enthusiasm. |
Upon close examination of the above example, we will be able to uncover the negative impacts of publication bias.
Firstly, if among the three studies, only study A gets published, it will skew the research in one direction and corroborate the findings of other studies which support the drug. These compelling findings will go on to influence practitioners who will regularly recommend this drug, thereby putting the patient at risk for its negative effects.
On the other hand, studies B and C, which would have captured the limitations of the drug, will not be made known to the world. Not just that, the non-publication would more often than not come across as a gap in the literature, prompting other researchers to explore it themselves, leading to a waste of time.
To publish their studies in journals, many resort to a practice known as data dredging or p-hacking. This typically involves looking for patterns in data to find an “attractive” result without clearly mentioning what the study’s primary hypothesis is. One of the ways data dredging occurs is when researchers run multiple tests on their data until they get statistically significant results.
Unfortunately, it is very difficult for us to avoid publication bias. So all you can do as a reader is be aware of the bias and its forms and read all your research with a pinch of salt.
5. Sponsorship Bias
It describes the tendency to modify or distort the methods and results of a study to favour the interests of the organisation funding the research.
Example: The Coca-Cola Company has funded several studies on weight management. Some of these studies, while highlighting the importance of exercise, have downplayed the role of calorie intake, potentially due to the company's financial interest in promoting sugary drinks. |
Companies can exert undue influence on various stages of the research process, right from formulating the research question to methodology and data collection. At times, their influence can also reach the stage of publication, during which they ask researchers to modify the data to make their product appear favourable. In many cases, companies have threatened to sue those researchers who have reported findings that are unfavourable to the company’s product.
While this is not the case for all collaborations, the few that exist pose a threat to academic freedom, interests, and rights.
As a reader, you can become more aware of sponsorship bias by looking at the “Conflict of Interest," "Disclosure,” or “Acknowledgements” sections. If the research is funded by grants, companies, or non-profit organisations, it is most likely to be mentioned in this section. With this knowledge, you will be better able to examine the research.
Research is an integral part of science, and even the most meticulous studies can be subject to unconscious bias. In this article, we looked at some common biases that can affect research design, data collection, and even publishing. It is critical for us, as informed information consumers, to understand that no study is flawless. However, understanding these biases allows us to evaluate the strengths and limitations of any research better, resulting in more educated decisions based on evidence rather than just conclusions.
References Smith, J., & Noble, H. (2014). Bias in research. Evidence Based Nursing, 17(4), 100–101. https://doi.org/10.1136/eb-2014-101946 |