Unless otherwise stated, the material in this guide is from The Learning Centre at Centennial College. Content has been adapted for the NWP Learning Commons in March 2022. This work is licensed under a Creative Commons BY 4.0 International License.
Hypothesis Testing
What is hypothesis testing?
Hypothesis testing in statistics is a way to test sample(s) with a population parameter. You can use hypothesis testing to test whether the data you gathered is a meaningful result. More specifically, how probable your sample data is compared to population parameters or other samples.
The null hypothesis, (\(H_0\)), represents the status quo. This means that the evidence collected is not strong enough to warrant a change.
The alternative hypothesis, (\(H_1\) or \(H_A\)), represents what is required to make a change to the status quo. That means there is statistical evidence using probability that it is different from the status quo.
For example, you believe your internet is slower than the average. You decide to collect the data on your connection speed every day for a month. The null hypothesis would be that there is no significant difference between your connection speed and the average, while the alternative hypothesis would be that your internet is slower.
So what is considered convincing evidence? To consider what evidence is convincing, we use a test statistic to decide whether to reject the null hypothesis or not. The distribution is divided up into two regions, one to reject the null hypothesis and one that remains the same. These regions are determined by the significance level or critical value.
The test statistic depends on the distribution. For normal or approximately normal distributions, we can use the z- or t-statistics.
The significance level, or critical value, splits the rejection region from the non-rejection region. For example, a significance level \(\alpha =0.05) in a two-tailed test for a normal approximation results in a critical value of 1.96 (found using the z-table).
Two-tailed test: The diagram above represents a two-tailed hypothesis test. This means that there is a rejection region at each of the two extreme ends of the distribution. The null and alternative hypotheses are stated as the following when comparing them to the population mean:
\begin{align} H_0 &= \mu \\ H_1 &\neq \mu \end{align}
One-tailed test:
There are two different one-tailed tests: one where the rejection region is in the upper extreme of the distribution, and the other the lower extreme.
The upper one-tailed test is usually stated:
\begin{align} H_0 &\leq \mu \\ H_1 &> \mu \end{align}
The lower one-tailed test:
\begin{align} H_0 &\geq \mu \\ H_1 &< \mu \end{align}
In some text, the \(\leq\) and \(\geq\) signs are replaced with the \(=\) sign.
Making a decision. The decision during hypothesis testing is whether to keep the status quo or if there is evidence to consider the alternative hypothesis. If the test statistic falls inside the rejection region, then there is evidence to reject the status quo. This does not mean that the alternative hypothesis is the true population parameter, just that there is evidence that this sample is different from the population parameter.
Example 1: A formal hypothesis test is to be conducted using the claim that the mean height of men is equal to 174.1 cm. What are the null and alternative hypotheses?
Solution: Since the test is to see if the population mean is equal to 174.1 cm, any mean that is significantly higher or lower will reject the claim. This is a two-tailed test because there are rejection regions in both the upper and lower extremes.
The null hypothesis or status quo is that the mean height of men is equal to 174.1 cm.
\[H_0=174.1\,cm\]
The alternative hypothesis is that there is evidence that there is a mean height statistically higher or lower than 174.1 cm.
\[H_1\neq 174.1\,cm\]
Example 2: Fewer than 95% of adults have a cell phone. In a poll of 1128 adults, 87% said they have a cell phone.
Solution: In this example, the status quo is that fewer than 95% of adults have a cell phone. The sample of 1128 adults shows a mean that may be lower. You want to test if this sample mean is significantly lower enough to warrant a change to the population mean of 95%.
You are performing a lower one-tailed test because you want to show this sample is significantly lower. A value significantly higher does not challenge this claim for this test. Therefore, the null and alternative hypotheses are (note that the population mean, 95%, is used and not the sample mean):
\begin{align} H_0 &\geq 95\% \\ H_1 &< 95\% \end{align}
Example 3: A survey of online reports shows that 68% of college administration staff believe that their online education courses are as good as or superior to those that utilize traditional face-to-face instruction. You collect a survey that shows that 75% of college administration staff make such a claim.
Solution: You are testing that your sample is showing a significant increase to the original claim. This is an upper one-tailed test since the rejection region of the claim will be consist of values that are statistically higher.
\begin{align} H_0 &\leq 68\% \\ H_1 &> 68\% \end{align}
The significance level \(\alpha\) for a hypothesis test is the probability used as the cutoff for what constitutes significant evidence against the null hypothesis.
For example, a significance level of \(\alpha=0.05\) in an upper one-tailed test in a normal distribution represents the following rejection region:
The critical value of \(z=1.645\) was found using the z-table looking up the top 5% of the normal distribution.
However, a significance level of \(\alpha=0.05\) during a two-tailed test means that the upper and lower rejection regions total 5%. That means on each side there is a rejection region of 2.5% or 0.025.
The critical value \(z=\pm 1.96\) was found by looking for the upper and lower 2.5% of the z-table.
Note that determining if it is a one-tailed or two-tailed test when setting up the alternative hypothesis is important to setting up the rejection region and finding the critical value.
You may think, "Why don't I just make the rejection region as large as possible, so I can verify that it is different?" Or "Let's make the rejection region as small as possible, so the status quo remains." This is not the best method to claim something is statistically different because of the errors that are created. There are two types of errors: Type I and Type II.
Type I error: The mistake of rejecting the null hypothesis when it is actually true. \(\alpha\), same as the significance level, is the type I error. So \(\alpha=0.05\) means that there is a 5% chance of rejecting the status quo, when in fact the status quo was the right decision.
Type II error: The mistake of failing to reject the null hypothesis when it is actually false. A type II error is represented by the symbol \(\beta\) and can be calculated using the Power of a Test.
As a result, there are four outcomes possible. Two of those outcomes are correct, and two are errors.
\(\alpha\) and \(\beta\) are inversely related (as one increases, the other decreases). Therefore, it is impossible to reduce both errors to 0. In an actual study, you will be choosing your own significance level, so keep type I and II errors in mind.
Example 1: There is a medicine that claims to cure a disease. Describe the type I and II errors.
| Null Hypothesis | \(H_0\) is true | \(H_0\) is false |
| Medicine cures the disease | The drug works and is not rejected. | Type I error: the medicine cures the disease, but the drug is rejected. |
| Medicine does not cure the disease | Type II error: the medicine does not cure the disease, but is not rejected. | The drug does not work and is rejected. |
Example 2: You decide to attend an online lecture because it will discuss a topic you are not familiar with.
| Null Hypothesis | \(H_0\) is true | \(H_0\) is false |
| You attend lecture | You attend lecture and it contains something you do not know. | Type I error: you attend the lecture but the lecture does not discuss the topic you do not know. |
| You do not attend lecture | Type II error: you miss the lecture but the topic you are not familiar with was discussed. | You miss the lecture and the topic was not discussed. |
Each possibility contains a probability value depending on the information provided.