One of the most common questions students, researchers, and professionals ask when they begin analyzing their data is deceptively simple:
What statistical test should I use?
It sounds like there should be a quick answer. If you have two groups, use one test. If you have three groups, use another. If you have survey data, use something else. In reality, choosing the right statistical analysis depends on several factors: the research question, the study design, the variables, the measurement scale, the sample size, the assumptions of the test, and the way the results need to be interpreted.
Choosing the wrong test can lead to incorrect conclusions, committee revisions, rejected manuscripts, or wasted time trying to interpret output that does not actually answer the research question. Choosing the right test, on the other hand, makes the analysis clearer, easier to defend, and much easier to write up.
This post provides a practical framework for choosing an appropriate statistical test for a dissertation, thesis, journal article, grant report, or applied research project.
Start With the Research Question, Not the Software
A common mistake is to begin with the software: SPSS, R, Stata, SAS, Excel, or Python. The software matters eventually, but it should not drive the choice of analysis.
The better starting point is the research question.
For example, consider these three questions:
- Do students in two different teaching methods differ in test scores?
- Is anxiety associated with depression?
- Does age, income, and education predict whether someone supports a policy?
These questions require different types of analyses because they ask different things. The first question asks about a group difference. The second asks about an association. The third asks about prediction and includes a binary outcome.
Before choosing a statistical test, ask:
- Am I comparing groups?
- Am I examining relationships between variables?
- Am I predicting an outcome?
- Am I testing change over time?
- Am I trying to control for other variables?
- Am I testing mediation, moderation, or a more complex model?
The statistical test should follow from the research question. If the test does not directly answer the research question, it is probably not the right test.
Identify Your Dependent Variable
The dependent variable is the outcome variable – the variable you are trying to explain, compare, or predict.
This is often the most important factor in choosing the right statistical test.
Ask: What kind of variable is my outcome?
Common possibilities include:
| Type of outcome | Example | Common analyses |
|---|---|---|
| Continuous | Test score, income, depression score, age | t-test, ANOVA, correlation, linear regression |
| Binary | Yes/no, pass/fail, employed/unemployed | Chi-square test, logistic regression |
| Categorical with more than two groups | Political party, diagnosis category, program type | Chi-square test, multinomial logistic regression |
| Ordinal | Likert-type rating, satisfaction category, severity level | Mann-Whitney U, Kruskal-Wallis, ordinal logistic regression |
| Count | Number of visits, number of symptoms, number of incidents | Poisson regression, negative binomial regression |
| Time-to-event | Time until relapse, time until graduation, survival time | Survival analysis, Cox regression |
Many errors in statistical analysis happen because the outcome variable is treated as the wrong type. For example, a yes/no outcome should usually not be analyzed with ordinary linear regression. A count outcome with many zeros may require a different model than a normally distributed continuous outcome.
Identify Your Independent Variable or Predictor
After identifying the outcome, look at the independent variable or predictor.
Ask:
- Is the predictor categorical or continuous?
- How many groups or levels does it have?
- Are the groups independent?
- Is the same person measured more than once?
- Are there multiple predictors?
- Do I need to control for covariates?
For example, if you are comparing mean depression scores between men and women, the independent variable is categorical with two groups. A t-test may be appropriate if assumptions are met.
If you are comparing mean depression scores across three therapy groups, the independent variable is categorical with three groups. ANOVA may be appropriate.
If you are predicting depression scores from age, social support, and income, multiple linear regression may be appropriate.
If you are predicting whether a person has clinically significant depression, logistic regression may be more appropriate.
Decide Whether You Are Comparing Groups or Examining Relationships
A helpful way to simplify test selection is to determine whether the research question is mainly about differences or relationships.
Questions About Group Differences
Group-difference questions ask whether one group differs from another group on an outcome.
Examples:
- Do men and women differ in job satisfaction?
- Do three treatment groups differ in anxiety scores?
- Do students’ scores improve from pretest to posttest?
- Are outcomes different across hospitals, schools, or departments?
Common tests for group differences include:
- Independent-samples t-test
- Paired-samples t-test
- One-way ANOVA
- Repeated-measures ANOVA
- ANCOVA
- MANOVA
- Mann-Whitney U test
- Wilcoxon signed-rank test
- Kruskal-Wallis test
- Chi-square test of independence
The right choice depends on the number of groups, whether the groups are independent, the outcome variable, and whether assumptions are met.
Questions About Relationships
Relationship questions ask whether variables are associated with each other.
Examples:
- Is age associated with job satisfaction?
- Is anxiety correlated with depression?
- Does social support predict burnout?
- Do income, education, and age predict health status?
Common tests for relationships include:
- Pearson correlation
- Spearman correlation
- Simple linear regression
- Multiple linear regression
- Logistic regression
- Ordinal logistic regression
- Poisson or negative binomial regression
- Structural equation modeling
Again, the correct choice depends heavily on the type of outcome variable and the research question.
Consider Whether Your Groups Are Independent or Paired
Another important issue is whether the observations are independent.
Independent Groups
Independent groups are separate groups of people or units. For example:
- Men vs. women
- Treatment group vs. control group
- Students from School A vs. students from School B
If the same person appears in only one group, the groups are usually independent.
For two independent groups with a continuous outcome, an independent-samples t-test may be appropriate. For three or more independent groups, ANOVA may be appropriate.
Paired or Repeated Measures
Paired or repeated-measures data occur when the same person or unit is measured more than once.
Examples:
- Pretest and posttest scores for the same students
- Patient symptoms before and after treatment
- Monthly sales for the same business
- Repeated surveys from the same participants over time
For two time points with a continuous outcome, a paired-samples t-test may be appropriate. For more than two time points, repeated-measures ANOVA or a mixed-effects model may be needed.
Treating paired data as independent data is a common mistake. It can produce incorrect standard errors and misleading p-values.
Match the Test to the Measurement Level
Many statistical tests assume that the outcome variable is measured at a certain level.
A continuous variable, such as age or income, can often be analyzed using t-tests, ANOVA, correlation, or linear regression. A binary variable, such as yes/no or pass/fail, often requires chi-square tests or logistic regression. An ordinal variable, such as a Likert-type response, may require special consideration depending on the number of categories, distribution, sample size, and how the variable is used.
This is especially important in dissertation research because many studies use survey instruments. Some survey scales are created by summing or averaging multiple Likert-type items. In practice, those scale scores are often treated as approximately continuous, especially when they have many possible values and acceptable reliability. However, individual Likert items with only a few response options may be better treated as ordinal.
The important point is not that there is always one perfect answer. The important point is that your decision should be defensible.
Think About the Number of Predictors
A simple statistical test may be enough when there is only one independent variable and one outcome. However, many research questions involve multiple predictors.
For example:
Research Question: Do age, years of experience, workload, and perceived supervisor support predict burnout?
This question would usually require a regression model, not a series of separate correlations. A multiple regression model allows you to examine the relationship between each predictor and the outcome while controlling for the other predictors.
Similarly, if the outcome is binary, such as whether someone left a job or stayed, logistic regression would often be more appropriate than linear regression.
Regression is especially useful when the goal is not merely to show whether two variables are associated, but to estimate whether a predictor matters after accounting for other variables.
Understand the Role of Covariates and Control Variables
Many studies include covariates or control variables.
Examples include:
- Age
- Gender
- Education
- Baseline score
- Income
- Years of experience
- Study site
- Prior exposure
- Treatment dosage
Including covariates can help address alternative explanations, reduce error variance, or adjust for baseline differences. But covariates should not be added casually. Each covariate should have a clear rationale based on the research question, prior literature, theory, or study design.
For example, if you are comparing post-treatment anxiety scores between a treatment group and a control group, and you also have baseline anxiety scores, ANCOVA may be appropriate. The baseline score can be included as a covariate so that the group comparison adjusts for initial differences.
However, adding too many covariates with a small sample can create unstable models. In dissertation research, this is a frequent problem: the proposed model may include many predictors, but the final sample size may not support that level of complexity.
Check the Assumptions of the Test
Most statistical tests have assumptions. These assumptions are not technical details to ignore; they affect whether the results can be trusted.
Common assumptions include:
- Independence of observations
- Normality of residuals
- Homogeneity of variance
- Linearity
- Absence of severe outliers
- Absence of multicollinearity
- Adequate expected cell counts
- Correct model specification
For example, an independent-samples t-test assumes that the observations are independent, the outcome is approximately normally distributed within groups, and the variances are reasonably similar across groups. Linear regression assumes linearity, independent errors, reasonable normality of residuals, and homoscedasticity, among other things.
In practice, assumptions are not always perfectly met. The question is whether violations are severe enough to make the chosen test inappropriate. Sometimes a test is robust to minor violations. Sometimes a transformation, nonparametric test, robust standard errors, or different model is needed.
Assumption testing should be connected to the analysis plan. It should not be an afterthought performed only after the results are already written.
Consider Sample Size
Sample size affects both statistical power and the stability of the model.
A small sample may be adequate for a simple descriptive analysis but inadequate for a complex regression model with many predictors. Sparse data can be especially problematic for logistic regression, chi-square tests, and models with many categories.
For example, suppose a dissertation study asks whether trauma history predicts elevated symptoms, but almost everyone in the sample has a trauma history. Even if the research question is reasonable, the data may not contain enough contrast to estimate the effect reliably. The analysis may need to be simplified, treated as exploratory, or reported with caution.
Small sample size does not automatically make analysis impossible. But it does limit what can be estimated, how precise the estimates are, and how confidently the results can be interpreted.
Common Research Questions and Possible Statistical Tests
The table below gives a general starting point. It is not a substitute for reviewing the design, data, assumptions, and hypotheses, but it can help narrow the possibilities.
| Research question | Example | Possible test or model |
|---|---|---|
| Are two independent groups different on a continuous outcome? | Do two teaching methods differ in exam scores? | Independent-samples t-test |
| Are three or more independent groups different on a continuous outcome? | Do three programs differ in satisfaction scores? | One-way ANOVA |
| Did the same participants change over time? | Did stress scores decrease from pretest to posttest? | Paired-samples t-test |
| Did the same participants change across three or more time points? | Did symptoms change from baseline to 3, 6, and 12 months? | Repeated-measures ANOVA or mixed-effects model |
| Are two categorical variables associated? | Is employment status associated with treatment group? | Chi-square test of independence |
| Are two continuous variables associated? | Is age associated with income? | Pearson correlation |
| Are two ordinal or non-normal variables associated? | Is rank satisfaction associated with rank stress? | Spearman correlation |
| Does one continuous predictor predict a continuous outcome? | Does workload predict burnout? | Simple linear regression |
| Do multiple predictors predict a continuous outcome? | Do age, workload, and support predict burnout? | Multiple linear regression |
| Do predictors estimate a yes/no outcome? | Do age and income predict whether someone graduates? | Logistic regression |
| Do predictors estimate an ordinal outcome? | Do predictors estimate low, medium, or high satisfaction? | Ordinal logistic regression |
| Is the outcome a count? | What predicts number of absences? | Poisson or negative binomial regression |
| Are there latent constructs or complex pathways? | Does stress mediate the relationship between workload and burnout? | Mediation analysis or structural equation modeling |
The key word in the final column is possible. A test that appears appropriate from a table may still be inappropriate after reviewing the actual dataset.
Avoid Choosing Tests Based Only on What Is Familiar
Many people choose a statistical test because they have used it before, because their software makes it easy to run, or because a committee member mentioned it. This can be risky.
For example:
- Running multiple t-tests instead of ANOVA can inflate Type I error.
- Using linear regression for a binary outcome can produce inappropriate predictions and standard errors.
- Using chi-square with very small expected cell counts can produce unreliable results.
- Ignoring repeated measures can violate independence assumptions.
- Treating a highly skewed count outcome as normally distributed can lead to poor model fit.
- Using a complex model with a very small sample can produce unstable estimates.
The best analysis is not necessarily the most advanced analysis. It is the analysis that best fits the research question, design, variables, and data.
Dissertation-Specific Considerations
Choosing a statistical test for a dissertation has some additional complications.
A dissertation analysis must usually satisfy several audiences at once:
- The analysis must answer the research questions.
- The analysis must align with the hypotheses.
- The analysis must be consistent with the proposal or approved methodology.
- The analysis must be understandable to the committee.
- The analysis must be reportable in APA format or the required institutional style.
- The analysis must be defensible if the data do not look exactly as expected.
This is why it is useful to think about the analysis before collecting data. The best time to choose the statistical test is during the design and proposal stage, not after the dataset has already been collected.
That said, many dissertation students only discover problems after data collection. The sample may be smaller than expected. A variable may be highly skewed. A category may have too few cases. A scale may have poor reliability. Some participants may have incomplete data. The final dataset may not support the originally proposed analysis.
When that happens, the goal is not to force the original test no matter what. The goal is to find a defensible analysis that answers the research question as well as the data allow.
Examples
Example 1: Comparing Two Groups
Suppose your research question is:
Do students who received online tutoring differ in final exam scores from students who did not receive tutoring?
The outcome is final exam score, which is continuous. The predictor is tutoring group, which has two independent groups: tutoring vs. no tutoring.
A reasonable starting point is an independent-samples t-test.
However, you would still need to consider assumptions, sample size, outliers, and whether other variables should be controlled. If the groups differed strongly at baseline, a regression or ANCOVA approach might be better.
Example 2: Predicting a Continuous Outcome
Suppose your research question is:
Do workload, supervisor support, and years of experience predict burnout among nurses?
The outcome is burnout score, which is continuous. There are multiple predictors.
A reasonable starting point is multiple linear regression.
This allows you to examine whether each predictor is associated with burnout while controlling for the others. You would also need to check linearity, residuals, multicollinearity, influential cases, and overall model fit.
Example 3: Predicting a Yes/No Outcome
Suppose your research question is:
Do age, prior experience, and training predict whether employees remain with the organization for at least one year?
The outcome is binary: stayed vs. did not stay.
A reasonable starting point is logistic regression.
Ordinary linear regression is usually not the best choice here because the outcome is not continuous. Logistic regression estimates the odds of the outcome as a function of the predictors.
Example 4: Analyzing Pretest/Posttest Data
Suppose your research question is:
Did participants’ anxiety scores decrease after a six-week intervention?
The same participants were measured before and after the intervention. The outcome is continuous, and there are two paired time points.
A reasonable starting point is a paired-samples t-test.
If there are multiple time points, multiple groups, missing repeated observations, or more complex structure, repeated-measures ANOVA or a mixed-effects model may be more appropriate.
When a More Advanced Model May Be Needed
Some studies require methods beyond t-tests, ANOVA, correlation, or basic regression.
For example:
- If students are nested within classrooms or schools, hierarchical linear modeling or mixed-effects modeling may be needed.
- If the study involves latent variables measured by multiple survey items, factor analysis or structural equation modeling may be appropriate.
- If the research question involves indirect effects, mediation analysis may be needed.
- If the effect of one variable depends on another variable, moderation analysis may be appropriate.
- If the outcome is a count with overdispersion, negative binomial regression may be better than Poisson regression.
- If the data involve time until an event occurs, survival analysis may be needed.
Advanced methods can be powerful, but they are not automatically better. They require appropriate data, adequate sample size, and careful interpretation.
How to Make the Final Decision
A practical decision process looks like this:
- State the research question clearly.
- Identify the dependent variable.
- Determine whether the outcome is continuous, categorical, ordinal, count, or time-to-event.
- Identify the independent variable or predictors.
- Determine whether the design involves independent groups, paired observations, repeated measures, or nested data.
- Decide whether the goal is comparison, association, prediction, or modeling a complex pathway.
- Check whether the proposed test matches the hypotheses.
- Review assumptions and sample size.
- Run the analysis.
- Interpret the results in plain language, not just statistical language.
The final step matters. A statistical test is only useful if the results can be explained clearly. A results section should not simply report output from SPSS, R, Stata, or another program. It should explain what the analysis showed, whether the hypothesis was supported, and what the findings mean in relation to the research question.
Common Mistakes to Avoid
Here are several common mistakes I see in dissertation and applied research projects:
Mistake 1: Choosing the Test Before Clarifying the Research Question
If the research question is vague, the analysis will usually be vague too. The test should be selected after the question is specific enough to analyze.
Mistake 2: Ignoring the Type of Outcome Variable
The dependent variable often determines the family of tests that should be considered. A continuous outcome, binary outcome, ordinal outcome, and count outcome usually require different approaches.
Mistake 3: Running Too Many Separate Tests
Running many separate tests can increase the chance of false positives and make the results harder to interpret. A more coherent model may be preferable.
Mistake 4: Using a Complex Model With Too Little Data
Advanced models can be appealing, but they need enough data. A model with many predictors and a small sample can produce unstable results.
Mistake 5: Treating Statistical Significance as the Only Thing That Matters
A p-value does not tell you whether an effect is large, important, or practically meaningful. Effect sizes, confidence intervals, descriptive statistics, and substantive interpretation also matter.
Mistake 6: Forgetting About the Write-Up
The analysis should be chosen with the final report in mind. If the results cannot be clearly explained, defended, and connected to the research questions, the analysis may need to be reconsidered.
Final Thoughts
Choosing the right statistical test is not about memorizing a single flowchart. Flowcharts and tables can help, but real research projects often involve complications: imperfect data, small samples, missing values, unusual distributions, multiple hypotheses, committee expectations, and practical constraints.
The right statistical test is the one that best matches the research question, the study design, the variables, the assumptions, and the purpose of the analysis.
For dissertation students, this decision is especially important because the statistical analysis must not only be technically appropriate, but also clearly justified and written in a way the committee can understand. For researchers and organizations, the same principle applies: the analysis should answer the question clearly, accurately, and defensibly.
If you are unsure which statistical test is appropriate for your dissertation, thesis, manuscript, survey, or applied research project, DK Statistical Consulting can help review your research questions, hypotheses, dataset, and analysis plan, and recommend an approach that fits your study.
A good analysis begins with the right question. The right test follows from there.