B1.5 Model Selection and F-Tests

When estimating the effect of a particular exposure, we have seen that it is important to include potential confounding variables in the regression model, and that failure to do so will lead to a biased estimate of the effect.

To assess if we have included important confounders in our model, we can run a statistical test to see if the extra coefficients are significantly contributing to the model fit. This is helpful for testing the overall inclusion of a categorical variable to a model (where some levels may have a significant association and other levels may not), or testing the addition of multiple variables to the model.

We can do this in R using the ‘linearHypothesis’ function within the package ‘car’:

install.packages(“car”)
library(car)

linearHypothesis(fit5, c(“bmi_fact1=0”, “bmi_fact3=0”, “bmi_fact4=0”))

Linear hypothesis test

Hypothesis:
bmi_fact1 = 0
bmi_fact3 = 0
bmi_fact4 = 0

Model 1: restricted model
Model 2: sbp ~ bmi_fact + ldlc + currsmoker

  Res.Df     RSS Df Sum of Sq      F   Pr(>F)    
1   4268 1310900                                 
2   4265 1303805  3      7095 7.7364 3.76e-05 ***
—
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The output reveals that the F-statistic for this joint hypothesis test (that all coefficients of the BMI variable are equal to 0) is about 7.7 and the corresponding p-value is <0.001.

Whilst the main regression output shows that not all levels of BMI group are significantly associated SBP, the F-test shows that overall this variable is significantly explaining variance in the model (F[3,4265]=7.74, p<0.001).

Question B1.5: Run an F-test to assess if both LDL-C and current smoking are jointly contributing to model fit (i.e. test both these coefficients at the same time).