Common Statistical Errors in Health Research Explained

Introduction

Health research is very important for health knowledge, patient care and for informing the design of health policies. Whether the analysis is performed in clinical trials or in epidemiological or healthcare evaluations, it is the statistics that provide the basis for meaningful conclusions drawn from the data collected. The reliability of research results is greatly influenced, however, on the use and understanding of statistical techniques. Statistics can give false information in even better-constructed studies if applied wrongly or if the basics of statistical design are not well known.

Errors in statistics can result in inaccurate interpretations, sub-optimal interventions, resource waste, in some instances, unhelpful choices of health care and so on. Scientists, doctors, and government officials use science to inform decisions that impact people’s health. Being familiar with common pitfalls is crucial for proper scientific rigour and validity of the results.

This article reviews common statistical mistakes made in health research such as the improper choice of statistical test, misinterpretation of p values, lack of consideration for confounding variables and exaggerated statements of cause-effect relationships. It also offers practical advice for minimising, if not eliminating, these errors and fostering strict scientific and ethical research practices based on well-founded biostatistical reasoning.

Role of statistical accuracy in health research

Statistics offer researchers means for summarizing data, finding patterns in data, testing hypotheses and making hypothetical estimates about the strength of relationships between variables. Appropriately applied, statistics can allow one to differentiate true vs random results.

Statistical techniques however, are not just mathematical procedures. They will need some reflective use, assumptions and contextual understanding. When statistical methods are misused, it can give a false sense of confidence in the findings and misrepresent scientific evidence, as well as fuel misinformation in the healthcare community. The current world of health research that has increasingly become a data-driven one demands a fundamental scientific skill: statistical literacy.

Lack of the appropriate use of Statistical Tests

An often committed mistake in health research is the inappropriate use of the wrong statistics tests. Each type of data, study design and research question has specific statistical test available. For instance, researchers might misapply parametric tests, which require data to be normally distributed, when the data are not normally distributed. On the other hand, conducting a series of tests without taking into account any of their assumptions can also introduce the opportunity for false results.

Inappropriate tests may be done because:

Failure to comprehend some of the assumptions of statistics.
Omitting consideration of distributions of data values.
Failure to consult properly with biostatisticians.
Use of familiar methods instead of appropriate methods.

How to Avoid This Error

Researchers should:

Express research questions prior to analysis.
Evaluate data for test assumptions.
Appropriate use of non-parametric alternatives when assumptions are not met.
Do not hesitate to ask for advice from the statistical experts when planning and analysing studies.
Record and explain the method used and why they were chosen.

Appropriate statistical planning prior to data collection can minimize the possibility for inappropriate analyses.

Misinterpretation of P-Values

The P-Value: what is it really?

One of the most common statistical errors in health-related studies is the misinterpretation of p-values. Although neatly used, p-values have proved to be misunderstood by the researchers and readers as well. A p-value is the likelihood of obtaining the data in the study, if the null hypothesis was true. Does not show someone the likelihood of the hypothesis being correct or incorrect.

However, many researchers err in believing that:

A p-value of less than 0.05 is evidence of a hypothesis.
P-values > 0.05 indicate no effect.
The lower the p value the greater the clinical significance.

These interpretations are wrong and can result in wrong conclusions.

The difficulty of achieving a satisfactory level of statistical significance

It is common for the traditional ‘5% cut-off’ of p < 0.05 to be regarded as the sharp boundary between meaningful and meaningless results. This practice can lead to researchers paying too much attention to statistical significance at the expense of practical significance.

For instance, a large study could show that there is a statistically significant difference between the two groups which is clinically irrelevant. On the other hand, even though they show relevant effects, statistically insignificant results can be obtained from a small study.

How to Avoid This Error

Researchers should:

Understand p-values in association with confidence intervals.
Don’t forget to report effect sizes when they can be done.
Pay attention to how the information is clinically relevant, and just as much to its non-clinical importance.
Refrain from expressing results as “significant” or “not significant.
Look at evidence in greater detail and over the course of a history period.

Be mindful that statistical significance is to be regarded as only one piece of evidence.

Ignoring Confounding Variables

What is meant by confounding variables?

When an external independent factor has an impact on the exposure and the process or outcome being studied, it creates a spurious association, this is called ‘confounding’.

That’s because studies examining the possible connection between caffeine and heart disease can find a connection, even if it’s positive. However, if there are increasing rates of smoking among people who use coffee then smoking may be the actual reason for this observed association. Without taking confounding factors into consideration, estimates can be biased and conclusions can be wrong.

Potential Problems to watch for:Potential Pitfall(s)

Ignoring confounders may:

Exaggerate true associations.
Hide genuine effects.
Produce false relationships.
Misguide healthcare interventions.

Confounding will cause many misconceptions over the years in medicine, especially observational studies.

Advice to prevent this error.

Confounding can be minimized by research by:

Recognising possible confounders at the study design stage.
Randomisation in clinical trials.
Applying stratification techniques.
Using multiple regression analysis.
Conducting sensitivity analyses.

Good planning and reporting provide greater accuracy in observable relationships.

Overstating Causal Relationships

Association doesn’t equal causation.

Health researchers often incorrectly interpret associations as being indicative of causation. Relationships are frequently discovered in observational studies, but an observed relationship does not necessarily imply causation.

Researchers can, for example, note that people who are physically active have less chronic disease. The positive effects of exercise for health may be confounded by other factors associated with better health, like nutrition, income, education, and health care quality.

Why This Matters

Exaggerated causal statements might:

Mislead healthcare providers.
Misinterpret and misrepresent public health guidelines.Distort and misrepresent public health guidance.
Set false hopes for patients.
Debauch honesty in scientific research.

This issue is often exaggerated by presenting correlative results as causal in headlines and news reports.

How to Avoid This Error

Researchers should:

Be sure to separate association from causation.
Practice avoiding ambiguous language and terms.
Discuss alternative explanations.
Understand the relative merits and weaknesses of the study design.
Use a powerful cause and effect statement for a suitable experimental investigation.

Clear communication fosters trust and the credibility of science.

Techniques of Multiple Comparisons, and Data Dredging

The 1st figure stands for the chance of obtaining false-positive results.

The greater number of statistical tests done on the same set of data the more likely it will be that a result will be significant by chance. A problem that arises in this case is referred to as the multiple comparisons problem. Doing multiple testing on many hypotheses at once may produce false-positive positive results that seem convincing but are simply chance occurrences. In cases where researchers attempt to analyze data in multiple ways until statistically significant results are found it is known as data dredging, or “p-hacking.

How to Avoid This Error

Researchers should:

Formulate prior to analysis.
Minimize guesswork testing.
Use multiple comparisons if appropriate.
Know the difference between explorative analyses and confirmatory analysis.
Obey study arrangements that were predetermined.

Such procedures maintain the integrity of the statistical results.

Inadequate Sample Size

Problems with Small Samples

The number of samples is one of its most important characteristics affecting the reliability of study results. Smaller sample sizes limit the ability of the studies to show any true impact.

Small studies may:

Produce unstable estimates.
Generate false-negative findings.
Overestimate effect sizes.
Reduce reproducibility.

Meanwhile, overly large numbers could also identify nuisance differences that are of little importance.

How to Avoid This Error

Researchers should:

Run power analyses prior to data collection.
Don’t overestimate possible effect sizes.
Take account of drop-out.
Explain Choice of Sample Sizes in Study Reports.

Validity and credibility is enhanced through adequate sample planning.

Neglecting Missing Data

Why Missing Data Matters

Data are frequently missing from health investigations as a result of the loss of participants from the study, the failure of participants to complete questionnaires, laboratory errors, or follow-up failure. Leaving missing data out will also result in significant bias in study outcomes, and treating it incorrectly will lead to bias.

For instance, for studies assessing the effectiveness of a treatment, if patients who are admitted with severe illness are more likely to have dropped out of the study, analysis excluding severe illness patients may lead to a positive conclusion.

How to avoid this error

Researchers should:

Discuss missing data and why.
Report extent of missing information.
When appropriate, impute when it is justifiable to do so.
Conduct sensitivity analyses.
Reporting: Use accepted procedures.

The presentation of missing information in a transparent manner enhances trust in the results of the study.

Reliance on Statistical Software

Complex analyses are possible with modern statistical software which provides an opportunity for the researchers to handle analyses. But it’s not possible to judge if a selected method is suitable or if assumptions have been met with software.

While using a computer-based output solely, researchers may miss other important considerations:

Violated assumptions.
Data quality concerns.
Model misspecification.
Interpretation errors.

How to Avoid This Error

Researchers should:

Gain knowledge of the principles of selected analyses.
Test and confirm hunches before drawing conclusions.
Pursue statistical experience as needed.
Work with experienced biostatisticians.

While software is a wonderful teaching tool to provide assistance, rather than supplanting thinking.

Selective Reporting of Results

Publication bias and reporting bias

Researchers occasionally only give positive results or just results when they are statistically significant, and they neglect to report null or negative results. This practice is known to be biased, which can lead to a bias in the published scientific literature. The selection of information can lead reviewers to view evidence as being more solid and convincing than is warranted.

Don’t fall into this trap

Researchers should:

Provide all pertinent results.
Follow pre-registered protocols.
Adhere to reporting guidelines.
Report deviations from scheduled course.
Encourage transparency, reproducibility.

Fully reported increases the scientific credibility and enables the precise synthesis of evidence.

Developing a Culture of Statistical Integrity

Enhancing statistical practices must be more than just a technical matter. It also requires transparency, education and ethical accountability. Research institutions, journals, and the funding agencies are increasingly keen on statistical rigor. A number of organizations nowadays promote study preregistration, Open data, Replication studies and collaboration with a biostatistical advisor.

Researchers who do the same enhance scientific knowledge and improve the quality of health care information. There should also be more awareness on statistical literacy: Today’s health professionals need to critically assess research outcomes and use these findings appropriately in their future work.

Conclusion

While vital statistical analysis plays a crucial role in health research, its use and interpretation are essential. Consistent problems with identifying and using statistical tests, interpretation of p values, confounding variables, exaggerating the cause and effect relationships, lack of sample size and reporting problems can compromise the credibility of science.

With awareness of these potential pitfalls and a well developed understanding of the analytical practices, researchers can enhance the quality, credibility, and impact of their research. Statistically sound argument aids to more correct inferences, decisions concerning health care, and increased public confidence in science. With the escalation of volume and complexity of health data, statistical integrity continues to be vital to further medical knowledge and safeguard the health of people and communities around the world.