P-Values and the Null Hypothesis
Introduction
In this lesson, we'll learn about the relationship between P-values and the Null Hypothesis, and their role in designing an experiment.
Objectives
You will be able to:
- Understand and explain the null hypothesis, including its role in sound experiment design
- Understand, calculate, and interpret P-values
Understanding The Null Hypothesis
As we said previously, scientific experiments actually have 2 hypotheses:
Null Hypothesis: There is no relationship between A and B Example: "There is no relationship between this flu medication and a reduced recovery time from the flu".
The Null Hypothesis is usually denoted as
Alternative Hypothesis: The hypothesis we traditionally think of when thikning of a hypothesis for an experiment Example: "This flu medication reduces recovery time for the flu."
The Alternative Hypothesis is usually denoted as
P-Values and Alpha Values
No matter what you're experimenting on, good experiments come down down to one question: Is our p-value less than our alpha value? Let's dive into what each of these values represents, and why they're so important to experimental design.
P-value: The calculated probability of arriving at this data randomly.
If we calculate a p-value and it comes out to 0.03, we can interpret this as saying "There is a 3% chance that the results I'm seeing are actually due to randomness or pure luck".
An alpha value can be any value we set between 0 and 1. However, the most common alpha value in science is 0.05 (although this is somewhat of a controversial topic in the scientific community, currently).
If we set an alpha value of
When we conduct an experiment, our goal is calculate a p-value and compare it to our alpha value. If
In simple terms:
There are many different ways that we can structure a hypothesis statement, but they always come down to this comparison in the end. In normally distributed data, we calculate p-values from z-scores. This is done a bit differently with discrete data. We may also have One-Tail and Two-Tail tests.
A One-Tail Test is when we want to know if a parameter from our treatment group is greater than (or less than) a corresponding parameter from our control group.
Example One-Tail Hypothesis
"$H_a = \mu_1 < \mu_2 $ The treatment group given this weight loss drug will lost more weight on average than the control group that was given a competitor's weight loss drug
$ H_o = \mu1 >= \mu_2$ The treatment group given this weight loss drug will not lose more weight on average than the control group that was given a competitor's weight loss drug".
A Two-Tail Test is for when we want to test if a parameter falls between (or outside of) a range of two given values.
Example Two-Tail Hypothesis
Use the Charts!
The following charts are designed to help you remember how to phrase, set up, and evaluate hypotheses for any experiment. Note that before you can know which chart to use, you need to define if your experiment is with Continuous or Discrete data, and if your hypothesis test is a One-Tail or a Two-Tail test!
Charts for Continuous Data
![]() |
![]() |
---|
Charts for Discrete Data
![]() |
![]() |
---|
Important Note
Do Not make the mistake of thinking you need to memorize each of the charts above. Instead, focus on understanding the differences between them. You'll find each of the following charts stored as an image within the corresponding github repo for this lesson.
Download these charts and store them on your machine for when you need them!
What Does an Experiment Really Prove?
You may be wondering why we need Null Hypothesis at all. This is a good question. It has to do with being honest about what an experiment actually proves.
Scientists use the Null Hypothesis so that we can be very specific in our findings. This is because a successful experiment doesn't actually prove a relationship between our dependent and independent variable. Instead, it just proves that we do not have enough evidence to convincingly believe there is no relationship between the dependent and the independent variable. There can always be a lurking variable behind the scenes that is actually responsible for the relationship between our two variables--its almost impossible to cover every possible angle. However, a successful experiment where our p-value is less than our alpha value (typically,
The Null Hypothesis Loves You and Wants You To Be Happy
We've covered a lot about the null hypothesis and how it's used in experiments in this lesson, but there's a lot more to learn about it!
Read the following article, The Null Hypothesis Loves You and Wants You To Be Happy. This does an excellent job of explaining why the concept of the Null Hypothesis is crucial to good science.
Summary
In this lesson, you learned about the relationship between P-values and the Null Hypothesis. Now let's move on and see how group sizes affect our tests!