Back to Course

FoSSA: Fundamentals of Statistical Software & Analysis

0% Complete
0/0 Steps
  1. Course Information

    Meet the Teaching Team
  2. Course Dataset 1
  3. Course Dataset 2
  4. MODULE A1: INTRODUCTION TO STATISTICS USING R, STATA, AND SPSS
    A1.1 What is Statistics?
  5. A1.2.1a Introduction to Stata
  6. A1.2.2b: Introduction to R
  7. A1.2.2c: Introduction to SPSS
  8. A1.3: Descriptive Statistics
  9. A1.4: Estimates and Confidence Intervals
  10. A1.5: Hypothesis Testing
  11. A1.6: Transforming Variables
  12. End of Module A1
    1 Quiz
  13. MODULE A2: POWER & SAMPLE SIZE CALCULATIONS
    A2.1 Key Concepts
  14. A2.2 Power calculations for a difference in means
  15. A2.3 Power Calculations for a difference in proportions
  16. A2.4 Sample Size Calculation for RCTs
  17. A2.5 Sample size calculations for cross-sectional studies (or surveys)
  18. A2.6 Sample size calculations for case-control studies
  19. End of Module A2
    1 Quiz
  20. MODULE B1: LINEAR REGRESSION
    B1.1 Correlation and Scatterplots
  21. B1.2 Differences Between Means (ANOVA 1)
  22. B1.3 Univariable Linear Regression
  23. B1.4 Multivariable Linear Regression
  24. B1.5 Model Selection and F-Tests
  25. B1.6 Regression Diagnostics
  26. End of Module B1
    1 Quiz
  27. MODULE B2: MULTIPLE COMPARISONS & REPEATED MEASURES
    B2.1 ANOVA Revisited - Post-Hoc Testing
  28. B2.2 Correcting For Multiple Comparisons
  29. B2.3 Two-way ANOVA
  30. B2.4 Repeated Measures and the Paired T-Test
  31. B2.5 Repeated Measures ANOVA
  32. End of Module B2
    1 Quiz
  33. MODULE B3: NON-PARAMETRIC MEASURES
    B3.1 The Parametric Assumptions
  34. B3.2 Mann-Whitney U Test
  35. B3.3 Kruskal-Wallis Test
  36. B3.4 Wilcoxon Signed Rank Test
  37. B3.5 Friedman Test
  38. B3.6 Spearman's Rank Order Correlation
  39. End of Module B3
    1 Quiz
  40. MODULE C1: BINARY OUTCOME DATA & LOGISTIC REGRESSION
    C1.1 Introduction to Prevalence, Risk, Odds and Rates
  41. C1.2 The Chi-Square Test and the Test For Trend
  42. C1.3 Univariable Logistic Regression
  43. C1.4 Multivariable Logistic Regression
  44. End of Module C1
    1 Quiz
  45. MODULE C2: SURVIVAL DATA
    C2.1 Introduction to Survival Data
  46. C2.2 Kaplan-Meier Survival Function & the Log Rank Test
  47. C2.3 Cox Proportional Hazards Regression
  48. C2.4 Poisson Regression
  49. End of Module C2
    1 Quiz

Learning Outcomes

By the end of this section, students will be able to:

  • Explain when and how to use post hoc testing 
  • Explain the concept of multiple comparisons and be able to correct for it in their analysis
  • Apply extensions to the basic ANOVA test and interpret their results
  • Explain when and how to use repeated measures statistics

You can download a copy of the slides here: B2.1 ANOVA Revisited- Post Hoc Testing

B2.1 PRACTICAL: R

In the last module we looked at linear regression to analyse the effect of different explanatory variables on the SBP of the participants.

We are going to revisit this here and look at it through the lens of the ANOVA.

Let’s look at the association of mean SBP across BMI groups again. But now we want to use the ANOVA post-estimation options to compute multiple comparisons with Fisher’s least-significant difference (LSD) test, which does not adjust the confidence intervals or p-values.

We can use the ‘LSD.test()’ function of the ‘agricolae’ package to perform this test in R.

#define group (factor) variable & fit one-way ANOVA

white.data<-Whitehall_fossa

white.data$bmi_fact<-factor(white.data$bmi_grp4)

model1<- aov(sbp~bmi_fact, data=white.data) summary(model1)

                    Df  Sum Sq Mean Sq F value   Pr(>F)
bmi_fact       3    6451  2150.4   7.024 0.000104 ***
Residuals   4297 1315528   306.2

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
26 observations deleted due to missingness

#perform Fisher’s LSD
install.packages(“agricolae”)

library(agricolae)

print(LSD.test(model1, “bmi_fact”)) 

$statisticsMSerror   Df    Mean      CV306.1504 4297 130.727 13.3845$parameters

test p.ajusted   name.t ntr alpha

Fisher-LSD      none bmi_fact   4  0.05

$means

sbp      std    r      LCL      UCL Min Max Q25   Q50    Q75

1 126.6600 18.47294   50 121.8088 131.5112  86 176 115 125.5 136.75

2 129.5695 17.49645 1791 128.7589 130.3801  87 230 118 128.0 139.00

3 131.3661 17.59025 2084 130.6147 132.1176  88 218 119 130.0 142.00

4 133.2394 16.83856  376 131.4703 135.0084  94 190 122 132.0 143.00

$comparison

NULL

$groups

sbp groups

4 133.2394      a

3 131.3661      a

2 129.5695      b

1 126.6600      b

attr(,”class”)

[1] “group”

To interpret this output we look at the section headed ‘$groups’. The groups that have different characters listed beside them (i.e. ‘a’ or ‘b’) are significantly different.

  • Question B2.1: Review your output. What do you notice? What can you conclude about the data from these tests? Compare this to your output from the linear regression on the same data.
Answer

Answer B2.1

The output of the main ANOVA gives the same values as the previous tests on this relationship (to be expected!). The post hoc testing, in the table titled ‘$groups’, shows the same relationships between groups as seen in the comparisons to reference categories shown as part of the linear regression on these data. Do check this for yourself by comparing to your own results from exercise B1.3.

The advantage of this method is that all of the comparisons are shown in one table, so you can inspect any pairing you wish.

In this output, we can see that :

  • Group 4 and group 2 have significantly different mean SBP, since group 4 has a value of ‘a’ and group 2 has a value of ‘b’.
  • Group 4 and group 1 have significantly different mean SBP, since group 4 has a value of ‘a’ and group 1 has a value of ‘b’.
  • Group 4 and group 3 do NOT have significant different mean SBP, since they both have a value of ‘a’.
B2.1 PRACTICAL: Stata

In the last module we looked at linear regression to analyse the effect of different explanatory variables on the SBP of the participants.

We are going to revisit this here and look at it through the lens of the ANOVA.

Let’s look at the association of mean SBP across BMI groups again. But now we want to use the ANOVA post-estimation options to compute multiple comparisons with Fisher’s least-significant difference (LSD) , which does not adjust the confidence intervals or p-values.

We can use the post-estimation command ‘pwcompare’ to run this test. In Stata, the default mcompare(noadjust) corresponds to Fisher’s protected LSD.

anova sbp bmi_grp4

[output omitted]

pwcompare bmi_grp4, mcompare(noadjust) effects

  • Question B2.1: Review your output. What do you notice? What can you conclude about the data from these tests? Compare this to your output from the linear regression on the same data.
Answer

The output of the main ANOVA gives the same values as the previous tests on this relationship (to be expected!). The post hoc testing, in the table titled ‘Pairwise comparisons’, shows the same relationships between groups as seen in the comparisons to reference categories shown as part of the linear regression on these data. Do check this for yourself by comparing to your own results from exercise B1.3.

The advantage of this method is that all of the comparisons are shown in one table, so you can inspect any pairing you wish.

B2.1 PRACTICAL: SPSS

In the last module we looked at linear regression to analyse the effect of different explanatory variables on the SBP of the participants.

We are going to revisit this here and look at it through the lens of the ANOVA.

Select

Analyze >> Compare Means and Proportions >> One-Way ANOVA

Move SBP into the Dependant List and BMI grouping (bmi_grp4) into the Factor box as before.

Now in and additional step, click on the post hoc button on the right-hand side. You will see a new box open up with lots of options for post hoc tests. We are going to select the very first option in the ‘Equal Variances Assumed’ section ‘LSD’ which stands for Least Significant Difference.

Press continue and then OK to run the test.

Review your output. What do you notice? What can you conclude about the data from these tests? Compare this to your output from the linear regression on the same data.

Answer

The output of the main ANOVA gives the same values as the previous tests on this relationship (to be expected!). The post hoc testing, in the table helpfully titled ‘Multiple Comparisons’, shows the same relationships between groups as seen in the comparisons to reference categories shown as part of the linear regression on these data. Do check this for yourself by comparing to your own results from exercise B1.3.

The advantage of this method is that all of the comparisons are shown in one table, so you can inspect any pairing you wish.

👋 Before you go, please rate your satisfaction with this lesson

Ratings are completely anonymous

Average rating 5 / 5. Vote count: 1

No votes so far! Be the first to rate this post.

Please share any positive or negative feedback you may have.

Feedback is completely anonymous

Subscribe
Notify of
guest

1 Comment
Newest
Oldest Most Voted
Inline Feedbacks
View all comments
Sayed Jalal

When I click on link for getting copy of the presentation: B2.1 ANOVA Revisited- Post Hoc Testing
it goes to: https://login.canvas.ox.ac.uk page. Could you please add the presentation here

1
0
Questions or comments?x