Which Statistical Test Should I Use?

A Complete Guide to Choosing the Right Statistical Test for Medical Research

Choosing the correct statistical test is one of the most important decisions in any research study. The wrong test can lead to invalid conclusions, wasted resources, and manuscript rejection. Yet many researchers struggle with this decision, often defaulting to familiar tests even when they are not appropriate for their data.

This guide walks you through the decision-making process step by step, covering the most commonly used statistical tests in clinical and biomedical research.

The Four Questions That Determine Your Test

Every statistical test selection ultimately comes down to four key characteristics of your data:

1. What type of outcome variable do you have? Continuous variables (like blood pressure or weight) require different tests than categorical variables (like alive/dead or treatment response). Ordinal data (like pain scales) and time-to-event data (like survival time) each have their own set of appropriate tests.
2. How many groups are you comparing? Comparing one group to a known value, two groups to each other, or three or more groups each requires a different test.
3. Are your observations paired or independent? If you measure the same patients before and after treatment, your data is paired. If you compare separate groups of patients, your data is independent. This distinction is critical because using an independent test on paired data wastes statistical power.
4. Is your data normally distributed? Parametric tests (like the t-test and ANOVA) assume normal distribution, while non-parametric tests (like Mann-Whitney and Kruskal-Wallis) do not. For large samples, the Central Limit Theorem often makes parametric tests robust even with non-normal data.

Parametric vs. Non-Parametric Tests

This is one of the most common sources of confusion in medical statistics. Parametric tests assume that your data follows a specific distribution (usually normal). They are generally more powerful, meaning they are better at detecting real differences when they exist. Non-parametric tests make fewer assumptions about your data distribution and work with ranks rather than raw values.

When to Use the Chi-Squared Test

The chi-squared test is perhaps the most commonly used test in medical research for categorical data. It compares observed frequencies to expected frequencies and is used to test whether there is an association between two categorical variables. The main requirement is that expected cell counts should be at least 5 in each cell. When this assumption is not met, Fisher exact test should be used instead.

A common mistake is using the chi-squared test for paired data. If you measured the same patients at two time points (e.g., before and after treatment) on a binary outcome, you need McNemar test, not chi-squared.

Understanding ANOVA and Its Variants

Analysis of variance (ANOVA) tests whether means differ across three or more groups. The basic one-way ANOVA compares independent groups on a single factor. If you find a significant result, you must perform post-hoc tests (such as Tukey HSD or Bonferroni) to determine which specific groups differ.

Two-way ANOVA adds a second factor and can test for interaction effects. Repeated measures ANOVA handles situations where the same subjects are measured multiple times. Mixed ANOVA combines both between-subjects and within-subjects factors.

Survival Analysis

Time-to-event data requires special statistical methods because of censoring: some patients have not yet experienced the event by the end of the study. The Kaplan-Meier method estimates survival curves, and the log-rank test compares them between groups. Cox proportional hazards regression models the effect of multiple predictors on survival while adjusting for confounders.

Correlation and Regression

Correlation measures the strength and direction of association between two variables. Pearson correlation measures linear relationships and requires normally distributed data. Spearman correlation measures monotonic relationships and works with ordinal data or non-normal distributions.

Regression goes further by modeling the relationship and allowing prediction. Linear regression models a continuous outcome, logistic regression models a binary outcome, and Cox regression models time-to-event outcomes. Multiple regression includes several predictors simultaneously, allowing you to adjust for confounders.

Sample Size and Statistical Power

No statistical test can overcome an inadequately powered study. Before collecting data, you should perform a sample size calculation based on the test you plan to use, the expected effect size, and your desired power (typically 80% or 90%). Each test has its own sample size formula, and using the wrong formula is a common error in study design.

Multiple Comparisons Problem

When you perform multiple statistical tests, the probability of finding at least one false positive increases. If you run 20 tests at alpha = 0.05, you expect one to be significant by chance alone. Solutions include Bonferroni correction (divide alpha by the number of tests), Holm-Bonferroni (less conservative), and false discovery rate control (Benjamini-Hochberg procedure).

Common Mistakes to Avoid

1. Using a parametric test on clearly non-normal small-sample data. Check normality with Shapiro-Wilk test and Q-Q plots before deciding.
2. Ignoring the paired nature of data. Pre-post measurements in the same patients must use paired tests.
3. Not correcting for multiple comparisons. Running many tests without adjustment inflates your false positive rate.
4. Using chi-squared with small expected cell counts. Switch to Fisher exact test when any expected cell count is below 5.
5. Treating ordinal data as continuous. While sometimes acceptable for multi-point scales, this should be justified.
6. Performing ANOVA without post-hoc tests. A significant ANOVA only tells you that groups differ somewhere, not which groups differ.
7. Using correlation to imply causation. Correlation only shows association; establishing causation requires proper study design.

Reporting Your Results

Good statistical reporting includes the test name, test statistic, degrees of freedom (when applicable), p-value, and an effect size measure with a confidence interval. Many journals now require effect sizes (such as Cohen d, odds ratios, or hazard ratios) rather than relying solely on p-values.

Always state whether tests were one-sided or two-sided (two-sided is standard unless justified otherwise), and report exact p-values rather than just "p < 0.05." If you used any corrections for multiple comparisons, state which method you used.