Choosing the correct statistical test is one of the most crucial decisions in dissertation research. The wrong test can invalidate your findings, while the right test strengthens your conclusions. This guide provides a systematic framework for selecting appropriate statistical analyses.
Decision Framework
To choose the right statistical test, answer these four questions:
1. What is your research question? Are you comparing groups, examining relationships, or making predictions?
2. How many variables are involved? One variable (univariate), two variables (bivariate), or multiple variables (multivariate)?
3. What types of variables do you have? Categorical (nominal/ordinal) or continuous (interval/ratio)?
4. Do your data meet parametric assumptions? Normal distribution, homogeneity of variance, independence of observations?
Quick Reference Decision Table
| Research Goal | Variable Types | Test | |--------------|----------------|------| | Compare 2 group means | 1 categorical (2 groups), 1 continuous | Independent t-test | | Compare 3+ group means | 1 categorical (3+ groups), 1 continuous | One-way ANOVA | | Compare means with 2 factors | 2 categorical, 1 continuous | Two-way ANOVA | | Compare pre/post means | 1 continuous measured twice | Paired t-test | | Test association | 2 categorical | Chi-square test | | Measure relationship | 2 continuous | Correlation | | Predict continuous outcome | Multiple predictors, 1 continuous | Multiple regression | | Predict categorical outcome | Multiple predictors, 1 binary | Logistic regression |
Tests for Comparing Groups
Independent samples t-test: Compares means between two unrelated groups. Use when you have one categorical independent variable with exactly two levels and one continuous dependent variable.
Example: "Is there a difference in exam scores between students who received tutoring and those who didn't?"
Assumptions: Normal distribution in each group, homogeneity of variance, independent observations.
Paired samples t-test: Compares means within the same group at two time points or under two conditions. Use for repeated measures or matched designs.
Example: "Did students' anxiety levels change from before to after the intervention?"
Assumptions: Differences between pairs are normally distributed.
One-way ANOVA: Compares means across three or more groups. Use when your independent variable has more than two levels.
Example: "Do students in three different teaching methods (lecture, discussion, project-based) differ in achievement?"
Assumptions: Normality within groups, homogeneity of variance, independence. Follow up with post-hoc tests (Tukey, Bonferroni) to identify which groups differ.
Two-way ANOVA: Examines main effects and interaction effects of two independent variables on one dependent variable.
Example: "How do teaching method and student gender affect achievement, and is there an interaction?"
Provides three F-tests: main effect of factor A, main effect of factor B, and A×B interaction.
Repeated measures ANOVA: Compares means across three or more time points within the same participants.
Example: "How does student motivation change across the beginning, middle, and end of semester?"
Assumptions: Sphericity (equal variances of differences between all pairs of conditions).
MANOVA (Multivariate ANOVA): Compares means on multiple dependent variables simultaneously.
Example: "Do teaching methods affect both achievement and motivation (two DVs) differently?"
Advantages: Controls Type I error when testing multiple DVs; can detect patterns not visible in separate ANOVAs.
Tests for Relationships
Pearson correlation (r): Measures the strength and direction of linear relationship between two continuous variables.
Range: -1 to +1. Interpretation: weak (.1-.3), moderate (.3-.5), strong (.5+).
Example: "Is there a relationship between study hours and exam scores?"
Assumptions: Both variables normally distributed, linear relationship, no extreme outliers.
Spearman correlation (ρ): Non-parametric alternative to Pearson. Use for ordinal data or when normality assumption is violated.
Example: "Is there a relationship between class rank and satisfaction rating?"
Chi-square test of independence: Tests whether two categorical variables are associated.
Example: "Is there an association between gender and career choice?"
Assumptions: Expected frequency ≥5 in each cell, independent observations.
Effect size: Cramér's V (.1 = small, .3 = medium, .5 = large).
Fisher's exact test: Alternative to chi-square when expected cell frequencies are too small (any expected count <5). Common with small samples.
Tests for Predictions
Simple linear regression: Predicts a continuous outcome from one continuous predictor.
Example: "Can we predict exam scores from study hours?"
Reports R² (variance explained), regression equation (Y = a + bX), and significance of the predictor.
Multiple regression: Predicts a continuous outcome from multiple predictors.
Example: "Can we predict job performance from motivation, experience, and training?"
Types: Simultaneous (all predictors entered at once), hierarchical (predictors entered in theory-driven blocks), stepwise (statistical selection—use cautiously).
Key statistics: R², adjusted R², F-test, individual predictor β weights and p-values.
Logistic regression: Predicts a binary categorical outcome from one or more predictors.
Example: "What factors predict whether students graduate (yes/no)?"
Reports odds ratios, confidence intervals, and Nagelkerke R².
Hierarchical linear modeling (HLM)/Multilevel modeling: For nested data structures (e.g., students within classrooms within schools).
Use when you have data at multiple levels and ignoring the nesting would violate independence assumptions.
Checking Assumptions
Testing normality:
- Visual: Histograms, Q-Q plots
- Statistical: Shapiro-Wilk test (p > .05 = normal), Kolmogorov-Smirnov test
- Rule of thumb: Skewness and kurtosis between -2 and +2 are acceptable
Testing homogeneity of variance:
- Levene's test: p > .05 indicates equal variances
- If violated: Use Welch's t-test or Games-Howell post-hoc for ANOVA
Testing linearity (for regression):
- Scatterplots of residuals vs predicted values
- Should show random scatter, not patterns
Testing multicollinearity (for multiple regression):
- Variance Inflation Factor (VIF): Values >10 indicate problematic collinearity
- Correlation matrix: Predictors correlating >.8 may cause issues
Non-Parametric Alternatives
When parametric assumptions are violated, use these alternatives:
| Parametric Test | Non-Parametric Alternative | |-----------------|---------------------------| | Independent t-test | Mann-Whitney U | | Paired t-test | Wilcoxon signed-rank | | One-way ANOVA | Kruskal-Wallis H | | Repeated measures ANOVA | Friedman test | | Pearson correlation | Spearman correlation |
When to use non-parametric tests:
- Small sample sizes (n < 30) with non-normal data
- Ordinal outcome variables
- Severe violations of normality or homogeneity
- Presence of significant outliers you cannot remove
Limitations: Non-parametric tests have less statistical power than their parametric counterparts, meaning you may miss real effects.
Common Mistakes to Avoid
1. Choosing tests based on results: Select your tests based on your research design and data characteristics, not on which test gives you significant results.
2. Ignoring assumption violations: Always check and report assumption tests. Use robust alternatives when assumptions are violated.
3. Multiple testing without correction: Running many tests inflates Type I error. Use Bonferroni correction or control false discovery rate.
4. Confusing correlation with causation: Correlation does not imply causation—only experimental designs can establish causal relationships.
5. Dichotomizing continuous variables: Don't convert continuous variables to categories (e.g., high/low) unless theoretically justified—it loses information and power.
6. Overreliance on p-values: Report effect sizes and confidence intervals, not just statistical significance.
7. Using the wrong level of measurement: Treating ordinal data as interval or nominal data as ordinal leads to inappropriate analyses.
Reporting Your Analysis
A complete statistical report includes:
- Why you chose this test (research question and data characteristics)
- Assumption checks and any corrections made
- Full test statistics (e.g., t, F, χ², r) with degrees of freedom
- Exact p-values (to three decimal places, or p < .001)
- Effect sizes and their interpretation
- Confidence intervals where applicable
- Any post-hoc tests and their results
