A1.5: Hypothesis Testing

A t-test compares the mean of a sample against another value or sample mean.  Prior to performing a t-test we should think about whether the data are continuous, have been randomly sampled from a population, the variance is approximately equal and the distribution is approximately normal.

To test whether the variability of the data in each group is similar:

robvar chol, by(prior_t2dm)

If p-values are >0.05, then we can assume equal variances and continue with the t-test.

If variances are not equal, we can still perform a t-test, but a slightly different version called a Welch t-test.

To check the distribution of the data:

histogram chol

We can also check whether both groups follow a normal distribution:

histogram chol, by(prior_t2dm) normal

There are a number of different t-tests you can perform:

• A one sample t-test
• A two sample t-test
• A paired t-test

One sample t-test

A one-sample t-test is used to compare the sample mean to a specific value (or hypothesised population mean).

For example, does the mean cholesterol level in this sample differ from the national average of 5.55?

ttest chol == 5.55

In the output from Stata, you should be able to see the sample size, the mean, standard error, standard deviation and 95% confidence intervals for cholesterol. At the bottom of the output, in the centre, should be able to see a p-value for the result ‘Pr(|T| . |t|)’. This refers to whether the mean cholesterol level in this sample is equal to the national mean of 5.55. The results indicate that the sample mean of 5.51 is significantly different from the national average (p=0.009).

Two-sample t-test

A two-sampled t-test is used to compare means between two groups. Check how your variables are categorised – you may need to create a new, binary variable in order to perform the t-test.

Example 1:

A systolic blood pressure of over 130 is defined as high. If we want to test whether cholesterol levels differ if an individual has high systolic blood pressure or not, we can use the following code:

• Create a new binary variable for systolic blood pressure

gen h_sbp=1 if sbp>130 & sbp~=.

replace h_sbp=0 if h_sbp==.

label var h_sbp “high SBP”

label values h_sbp h_sbp

label define h_sbp 1 “High SBP” 0 “Not high SBP”

tab h_sbp, m

• Perform the t-test using new variable

ttest chol, by (h_sbp)

Example 2:

Does the mean total cholesterol level differ between people with and without type 2 diabetes?

ttest chol, by (prior_t2dm)

If you can’t assume equal variances, you can use either of the following commands:

ttest chol, by (prior_t2dm) welch

ttest chol, by (prior_t2dm) unequal

In the output from Stata, you should be able to see the sample size, the mean, standard error, standard deviation and 95% confidence intervals for cholesterol level in each group (diabetes: yes or no). At the bottom of the output, in the centre, should be able to see a p-value for the result ‘Pr(|T| > |t|) ‘ for the two-tailed p-value for the t-test. If the p-value for ‘Pr(|T| > |t|)’ is >0.05, there is no significant difference between the means of both groups. However, if the p-value is <0.05, then we infer that there is a statistically significant difference between the mean cholesterol levels in people with and without type 2 diabetes.

• See if you can perform a two-sample t-test to compare cholesterol levels between two other groups. 

To perform a t-test for each level of a categorical variable you can use the command ‘by’. To do this sort your data by the categorical variable and then perform the t-test:

sort death

by death: ttest chol, by (prior_t2dm)

The above command should perform two t-tests: One comparing mean cholesterol in those with and without type 2 diabetes, but only in those who died. The other comparing mean cholesterol in those with and without type 2 diabetes, but only in those who did not die.

Paired t-test

A paired t-test compares the means between two groups when there are paired observations. This will be covered in Module B2.5

Question A1.5a: Recode BMI into a binary variable so that one group has a BMI below 25, and the other group has a BMI of 25 and above. Perform a t-test to compare the mean SBP in those with BMI<25 and those with BMI≥. Answer the questions:

1. 1. What is the mean SBP where BMI <25 () ?
   2. What is the mean SBP where BMI ≥25 ()?
   3. What is the mean difference ()?
   4. What is the test statistic t ?
   5. What is 95% CI for the mean difference?
   6. What is the p-value for this t-test and what does it mean?

Question A1.5b: If a clinician has decided that a difference of at least 5 mmHg is considered a clinically worthwhile difference in blood pressure with regard to morbidity associated with high blood pressure, do you consider the result in A1.5a to be clinically significant?